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resulting rectangular wave is rounded off by a band-pass filter that suppresses har- 
monics of the carrier frequency. Thus the filter output is again sinusoidal, with an ampli- 
tude that is practically independent of the carrier amplitude at the receiver input (see 
Problem 2.42). 
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| 2.10 Noise in CW Modulation Systems 

Up to this point in our discussion we have focused attention on the characterization of 
continuous-wave (CW) modulation techniques, entirely from a deterministic perspective. 
In the remainder of the chapter, we study the effects of channel noise on the reception of 
CW modulated signals and thereby develop a deeper understanding of the behavior of 
analog communications. 

To undertake such a study we follow the customary practice by formulating two 
models: 

1. Channel model, which assumes a communication channel that is distortionless but 
perturbed by additive white Gaussian noise (AWGN). 

2. Receiver model, which assumes a receiver consisting of an ideal band-pass filter fol- 
lowed by an ideal demodulator appropriate for the application at hand; the band- 
pass filter is used to minimize the effect of channel noise. 

These simplifying assumptions are made in order to obtain a basic understanding of the 
way in which noise affects the performance of the receiver. Moreover, they provide a 
framework for the comparison of different CW modulation-demodulation schemes. 

Figure 2.33 shows the noisy receiver model that combines the above two assump- 
tions. In this figure, s{t) denotes the incoming modulated signal and w(t) denotes the 
channel noise. The received signal is therefore made up of the sum of s(t) and w(t); this is 
the signal that the receiver has to work on. The band-pass filter in the model of Figure 
2.33 represents the combined filtering action of the tuned amplifiers used in the actual 
receiver for the purpose of signal amplification prior to demodulation. The bandwidth of 
this band-pass filter is just wide enough to pass the modulated signal s(f) without distor- 
tion. As for the demodulator in the model of Figure 2.33, its details naturally depend on 
the type of modulation used. 



■ SIGNAL-TO-NOISE RATIOS: BASIC DEFINITIONS 

Let the power spectral density of the noise w(t) be denoted by N 0 /2, defined for both 
positive and negative frequencies; that is, N 0 is the average noise power per unit bandwidth 
measured at the front end of the receiver. We also assume that the band-pass filter in the 
receiver model of Figure 2.33 is ideal, having a bandwidth equal to the transmission band- 
width B T of the modulated signal 5(f) and a midband frequency equal to the carrier fre- 
quency /„. The latter assumption is justified for double sideband-suppressed carrier (DSB- 
SC) modulation, full amplitude modulation (AM), and frequency modulation (FM); the 
cases of single sideband (SSB) modulation and vestigial sideband (VSB) modulation require 
special considerations. Taking the midband frequency of the band-pass filter to be the 
same as the carrier frequency f c , we may model the power spectral density S N {f) of the 
noise n(t), resulting from the passage of the white noise w (t) through the filter, as shown 
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Figure 2.33 Receiver model. 
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in Figure 2.34. Typically, the carrier frequency f c is large compared to the transmission 
bandwidth B T . We may therefore treat the filtered noise n(t) as a narrowband noise rep- 
resented in the canonical form 

n[t) = n,(t) cos(2ir/ c r) - n Q (t) sin(2-irf c t) (2.79) 

where n t (t) is the in-phase noise component and n Q (t) is the quadrature noise component, 
both measured with respect to the carrier wave A c cos(2irf c t). The filtered signal x(t) avail- 
able for demodulation is denned by 

x(t) = s(t) + n(t) (2.80) 

The details of s(t) depend on the type of modulation used. In any event, the average noise 
power at the demodulator input is equal to the total area under the curve of the power 
spectral density S N {f ). From Figure 2.34 we readily see that this average noise power is 
equal to NoB T . Given the format of s(t), we may also determine the average signal power 
at the demodulator input. With the demodulated signal s(f) and the filtered noise (n(t) 
appearing additively at the demodulator input in accordance with Equation (2.80), we 
may go on to define an input signal-to-noise ratio, (SNR) h as the ratio of the average 
power of the modulated signal s{t) to the average power of the filtered noise n(t). 

A more useful measure of noise performance, however, is the output signal-to-noise 
ratio, (SNR)o, defined as the ratio of the average power of the demodulated message signal 
to the average power of the noise, both measured at the receiver output. The output signal- 
to-noise ratio provides an intuitive measure for describing the fidelity with which the de- 
modulation process in the receiver recovers the message signal from the modulated signal 
in the presence of additive noise. For such a criterion to be well defined, the recovered 
message signal and the corruptive noise component must appear additively at the demod- 
ulator output. This condition is perfectly valid in the case of a receiver using coherent 
detection. On the other hand, when the receiver uses envelope detection as in full AM or 
frequency discrimination as in FM, we have to assume that the average power of the filtered 
noise n(f ) is relatively low to justify the use of output signal-to-noise ratio as a measure of 
receiver performance. 

The output signal-to-noise ratio depends, among other factors, on the type of mod- 
ulation used in the transmitter and the type of demodulation used in the receiver. Thus it 
is informative to compare the output signal-to-noise ratios for different modulation- 
demodulation systems. However, for this comparison to be of meaningful value, it must 
be made on an equal basis as described here: 

88 The modulated signal s(t) transmitted by each system has the same average power. 
*- The channel noise w(t) has the same average power measured in the message band- 
width W. 
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Figure 2.34 Idealized characteristic of band-pass filtered noise. 
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Figure 2.35 The baseband transmission model, assuming a message signal of bandwidth W, 
used for calculating the channel signal-to-noise ratio. 



Accordingly, as a frame of reference we define the channel signal-to-noise ratio, (SNR) C , 
as the ratio of the average power of the modulated signal to the average power of channel 
noise in the message bandwidth, both measured at the receiver input. This definition is 
illustrated in Figure 2.35. 

For the purpose of comparing different continuous-wave (CW) modulation systems, 
we normalize the receiver performance by dividing the output signal-to-noise ratio by the 
channel signal-to-noise ratio. We thus define a figure of merit for the receiver as follows: 

(SNR)o 

Figure of merit = (z.sl) 
(MNKJc 

Clearly, the higher the value of the figure of merit, the better will the noise performance 
of the receiver be. The figure of merit may equal one, be less than one, or be greater than 
one, depending on the type of modulation used, which will become apparent from the 
discussion that follows. 



2.11 Noise in Linear Receivers 
Using Coherent Detection 

From Sections 2.2 and 2.3 we recall that the demodulation of an amplitude-modulated 
wave depends on whether the carrier is suppressed or not. When the carrier is suppressed 
we usually require the use of coherent detection, in which case the receiver is linear. On 
the other hand, when the amplitude modulation includes transmission of the carrier, de- 
modulation is accomplished simply by using an envelope detector, in which case the re- 
ceiver is nonlinear. In this section we study the effect of noise on the performance of a 
linear receiver. The more difficult case of a nonlinear receiver is deferred to Section 2.12. 

Consider the case of DSB-SC modulation Figure 2.36 shows the model of a DSB-SC 
receiver using a coherent detector. The use of coherent detection requires multiplication 
of the filtered signal x(t) by a locally generated sinusoidal wave cos(2ir/ c t) and then low- 
pass filtering the product. To simplify the analysis, we assume that the amplitude of the 
locally generated sinusoidal wave is unity. For this demodulation scheme to operate sat- 
isfactorily, however, it is necessary that the local oscillator be synchronized both in phase 
and in frequency with the oscillator generating the carrier wave in the transmitter. We 
assume that this synchronization has been achieved. 

The DSB-SC component of the filtered signal x(t) is expressed as 

s(t) = CA C cos(2irf J)m(t) (2.82) 

where A c cos(27r/ e t) is the sinusoidal carrier wave and m(t) is the message signal. In the 
expression for s(t) in Equation (2.82) we have included a system-dependent scaling factor 
C, the purpose of which is to ensure that the signal component s(t ) is measured in the same 
units as the additive noise component n(t). We assume that m{t) is the sample function of 
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Figure 2.36 Model of DSB-SC receiver using coherent detection. 



a stationary process of zero mean, whose power spectral density S M (f) is limited to a 
maximum frequency W; that is, W is the message bandwidth. The average power P of the 
message signal is the total area under the curve of power spectral density, as shown by 

,w 

P = S M (f) df (2.83) 
J-w 

The carrier wave is statistically independent of the message signal. To emphasize this 
independence, the carrier should include a random phase that is uniformly distributed over 
2-ir radians. In the defining equation for s(t) this random phase angle has been omitted for 
convenience of presentation. Using the result of Example 1.7 of Chapter 1 on a modulated 
random process, we may express the average power of the DSB-SC modulated signal 
component s(t) as C 2 A JP/2. With a noise spectral density of N 0 /2, the average noise power 
in the message bandwidth W is equal to WN 0 . The channel signal-to-noise ratio of the 
DSB-SC modulation system is therefore 

(SNR) c , DS£ = g|| (2.84) 

where the constant C 2 in the numerator ensures that this ratio is dimensionless. 

Next, we wish to determine the output signal-to-noise ratio of the system. Using the 
narrowband representation of the filtered noise n(t), the total signal at the coherent detec- 
tor input may be expressed as 

*<*) = *(*) + »(*> 

= CA C cos(2ir/ c f)m(f) + n,{t) cos(2ir/ e r) - n Q (t) sm(2nf c t) 

where tij(t) and «q(?) are the in-phase and quadrature components of n(t) with respect to 
the carrier. The output of the product-modulator component of the coherent detector is 
therefore 

v(t) = x(t) cos(2i7/ £ r) 
= jCAMt) + \n,(t) 

+ %[CAjn(t) + n,(t)] cos(4Trf c t) - \n Q (t) sin(4ir/ t; f) 

The low-pass filter in the coherent detector in Figure 2.36 removes the high-frequency 
components of v(t), yielding the receiver output 

y{t) = \CA c m(t) + in,(t) (2.86) 

Equation (2.86) indicates the following: 

1. The message signal m{t) and in-phase noise component n T (t) of the filtered noise nit) 
appear additively at the receiver output. 
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2. The quadrature component n Q (t) of the noise n(t) is completely rejected by the co- 
herent detector. 

These two results are independent of the input signal-to-noise ratio. Thus, coherent detec- 
tion distinguishes itself from other demodulation techniques in an important property: The 
output message component is unmutilated and the noise component always appears ad- 
ditively with the message, irrespective of the input signal-to-noise ratio. 

The message signal component at the receiver output is CA c m(t)/2. Therefore, the 
average power of this component may be expressed as C 2 A*P/4, where P is the average 
power of the original message signal m(t) and C is the system-dependent scaling factor 
referred to earlier. 

In the case of DSB-SC modulation, the band-pass filter in Figure 2.36 has a band- 
width B T equal to 2W in order to accommodate the upper and lower sidebands of the 
modulated signal s(f ). It follows therefore that the average power of the filtered noise n(t) 
is 2WN 0 . From the discussion of narrowband noise presented in Section 1.11, we know 
that the average power of the (low-pass) in-phase noise component n,(t) is the same as 
that of the (band-pass) filtered noise n{t). Since from Equation (2.86) the noise component 
at the receiver output is n,(t)/2, it follows that the average power of the noise at the receiver 
output is 

(i) 2 2WN 0 = !WN 0 

The output signal-to-noise for a DSB-SC receiver using coherent detection is therefore 

C 2 A 2 c P/4 



(SNR) 0> DSB-SC 



WN 0 /2 

(2.87) 

C 2 AlP 



2WN 0 

Using Equations (2.84) and (2.87), we obtain the figure of merit 

(SNR), 



(SNR), 



= 1 (2.1 

DSB-SC 



Note that the factor C 2 is common to both the output and channel signal-to-noise ratios, 
and therefore cancels out in evaluating the figure of merit. 

Following through the noise analysis of a coherent detector for SSB, we find that, 
despite the fundamental differences between it and the coherent detector for DSB-SC mod- 
ulation, the figure of merit is exactly the same for both of them; see Problem 2.49. 

The important conclusions to be drawn from the discussions presented in this section 
and Problem 2.49 are two-fold: 

1. For the same average transmitted or modulated signal power and the same average 
noise power in the message bandwidth, a coherent SSB receiver will have exactly the 
same output signal-to-noise ratio as a coherent DSB-SC receiver. 

2. In both cases, the noise performance of the receiver is exactly the same as that ob- 
tained by simply transmitting the message signal in the presence of the same channel 
noise. The only effect of the modulation process is to translate the message signal to 
a different frequency band to facilitate its transmission over a band-pass channel. 

Simply put, neither DSB-SC modulation nor SSB modulation offers the means for a trade- 
off between improved noise performance and increased channel bandwidth. This is a se- 
rious problem when high quality of reception is a requirement. 
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Noise in AM Receivers 
Envelope Detection 

The next noise analysis we perform is for an amplitude modulation (AM) system using an 
envelope detector in the receiver, as shown in the model of Figure 2.37. In a full AM signal, 
both sidebands and the carrier wave are transmitted, as shown by 

i(f) = A c \\ + k a m(t)] cos(277/ c f) (2.89) 

where A c cos(27r/ e t) is the carrier wave, raff) is the message signal, and k a is a constant 
that determines the percentage modulation. In the expression for the amplitude-modulated 
signal component s(t) given in Equation (2.89), we see no need for the use of a scaling 
factor, because it is reasonable to assume that the carrier amplitude A c has the same units 
as the additive noise component. 

The average power of the carrier component in the AM signal s(t) is A^/2. The 
average power of the information-bearing component A c kjn{t) cos(2-n/ c f) is A^klP/2, 
where P is the average power of the message signal m(t). The average power of the full 
AM signal s(t) is therefore equal to A 2 C (1 + k^P)/2. As for the DSB-SC system, the average 
power of noise in the message bandwidth is WN 0 . The channel signal-to-noise ratio for 
AM is therefore 

(SNRJcam - 2WNg (2.90) 

To evaluate the output signal-to-noise ratio, we first represent the filtered noise n(t) 
in terms of its in-phase and quadrature components. We may therefore define the filtered 
signal x(t) applied to the envelope detector in the receiver model of Figure 2.37 as follows: 

x(t) = i(f) + n(t) 

= \A C + AJkMt) + n,(t)] cos(277/ e f) - n B (t) sin^^f) 

It is informative to represent the components that comprise the signal x{t) by means of 
phasors, as in Figure 2.38ii. From this phasor diagram, the receiver output is readily ob- 
tained as 

y(t) = envelope of x(t) 

= {[A c + A c kMt) + ni (t)f + nl(t)} vl 

The signal y(t) defines the output of an ideal envelope detector. The phase of x(t) is of no 
interest to us, because an ideal envelope detector is totally insensitive to variations in the 
phase of x(t). 

The expression defining y(t) is somewhat complex and needs to be simplified in some 
manner to permit the derivation of insightful results. Specifically, we would like to ap- 
proximate the output y(t) as the sum of a message term plus a term due to noise. In general, 
this is quite difficult to achieve. However, when the average carrier power is large com- 
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Figure 2.38 (a) Phasor diagram for AM wave plus narrowband noise for the case of high car- 
rier-to-noise ratio. {&) Phasor diagram for AM wave plus narrowband noise for the case of low 
carrier-to-noise ratio. 



pared with the average noise power, so that the receiver is operating satisfactorily, then 
the signal term A c [\ + k a m(t)] will be large compared with the noise terms n r (t) and n Q (t), 
at least most of the time. Then we may approximate the output y(t ) as (see Problem 2.51): 



y(t) ~A C + A c k a m(t) + «i(f) 



(2.93) 



The presence of the DC or constant term A c in the envelope detector output y(t) of 
Equation (2.93) is due to demodulation of the transmitted carrier wave. We may ignore 
this term, however, because it bears no relation whatsoever to the message signal m(t). In 
any case, it may be removed simply by means of a blocking capacitor. Thus if we neglect 
the DC term A c in Equation (2.93), we find that the remainder has, except for scaling 
factors, a form similar to the output of a DSB-SC receiver using coherent detection. Ac- 
cordingly, the output signal-to-noise ratio of an AM receiver using an envelope detector 
is approximately 



(SNR) G>AM - 



2WN 0 



(2.94) 



Equation (2.94) is, however, valid only if the following two conditions are satisfied: 

1. The average noise power is small compared to the average carrier power at the 
envelope detector input. 

2. The amplitude sensitivity k a is adjusted for a percentage modulation less than or 
equal to 100 percent. 

Using Equations (2.90) and (2.94), we obtain the following figure of merit for amplitude 
modulation: 



(SNR) C 



(SNR) C 



1 + & 



(2.95) 



Thus, whereas the figure of merit of a DSB-SC receiver or that of an SSB receiver using 
coherent detection is always unity, the corresponding figure of merit of an AM receiver 
using envelope detection is always less than unity. In other words, the noise performance 
of a full AM receiver is always inferior to that of a DSB-SC receiver. This is due to the 
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wastage of transmitter power, which results from transmitting the carrier as a component 
of the AM wave. 



> Example 2.4 Single-Tone Modulation 

Consider the special case of a sinusoidal wave of frequency /„ and amplitude A„ as the 
modulating wave, as shown by 

m{t) = A m cos(2ir/„r) 

The corresponding AM wave is 

s(t) = A c [l + ix cos(2irf m t)] cos(2irfr) 

where y, = kji m is the modulation factor. The average power of the modulating wave m(t) is 
(assuming a load resistor of 1 ohm) 

P = jAi 

Therefore, using Equation (2.95), we get 



(SNR) 0 



(SNR) f 



\klAl, 

1 + \ klA 2 „ 

2 (2.96) 



2 + A 2 



When y, = 1, which corresponds to 100 percent modulation, we get a figure of merit equal 
to 1/3. This means that, other factors being equal, an AM system (using envelope detection) 
must transmit three times as much average power as a suppressed-carrier system (using co- 
herent detection) to achieve the same quality of noise performance. -4 

s Threshold Effect 

When the carrier-to-noise ratio is small compared with unity, the noise term dominates 
and the performance of the envelope detector changes completely from that just described. 
In this case it is more convenient to represent the narrowband noise n(t) in terms of its 
envelope r(t) and phase ip(t), as shown by 

n(t) = r(t) cos[2ir£f + (/-(f)] (2.97) 

The corresponding phasor diagram for the detector input x(t) = $(t) + n(t) is shown in 
Figure 2.38£>, where we have used the noise envelope as reference, because it is now the 
dominant term. To the noise phasor r(t) we have added a phasor representing the signal 
term A e [l + k a m(t)], with the angle between them being equal to the phase i//(t) of the 
noise n(t). In Figure 2.38b it is assumed that the carrier-to-noise ratio is so low that the 
carrier amplitude A c is small compared with the noise envelope r(t ), at least most of the 
time. Then we may neglect the quadrature component of the signal with respect to the 
noise, and thus find from Figure 2.38£> that the envelope detector output is 

y(t) = r(t) + A c cos[<MJ)] + AJt a m(t) cos[i/-(f)] (2.98) 

This relation reveals that when the carrier-to-noise ratio is low, the detector output has 
no component strictly proportional to the message signal m(t). The last term of the ex- 
pression defining y(t) contains the message signal m(t) multiplied by noise in the form of 
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cos[t/<(f)]. From Section 1.11 we recall that the phase \jj(t) of the narrowband noise n(t) is 
uniformly distributed over 2-ir radians. It follows therefore that we have a complete loss 
of information in that the detector output does not contain the message signal tn(t) at all. 
The loss of a message in an envelope detector that operates at a low carrier-to-noise ratio 
is referred to as the threshold effect. By threshold we mean a value of the carrier-to-noise 
ratio below which the noise performance of a detector deteriorates much more rapidly 
than proportionately to the carrier-to-noise ratio. It is important to recognize that every 
nonlinear detector (e.g., envelope detector) exhibits a threshold effect. On the other hand, 
such an effect does not arise in a coherent detector. 

A rigorous mathematical analysis of the threshold effect for the general case of an 
AM wave is beyond the scope of this book. In the next subsection we simplify matters by 
considering the case of an unmodulated carrier. Despite this simplification, we can still 
develop a great deal of insight into the threshold effect experienced in an envelope detector. 

General Formula for (SNR) 0 in Envelope Detection* 

Consider an envelope detector whose input signal is defined by 

x(t) = A c cos(27r/ c r) + n(t) (2.99) 

where A c cos(2Trf t t) is the unmodulated carrier and n(t) is the sample function of band- 
limited, zero-mean, white Gaussian noise N(f). The power spectral density of N(t) is 

wo- fy **\f- fe \*v (2 . 100) 

[0 otherwise 

Representing the narrowband noise n(t) in terms of its in-phase component «;(r) and 
quadrature component n Q (t), we may express the noisy signal at the detector input as 

x(t) = (A c + m(t)) cos(27r/ c r) - n Q (t) sm(2irf c t) (2.101) 

The noise components nAt) and n Q (t) are zero-mean, jointly Gaussian, mutually indepen- 
dent low-pass random processes with identical power spectral densities (see Equation 
1.101): 

S Nl (f) - S NQ (f) - | Q otherwise 1 ' 

For the problem at hand, the input signal consists of an unmodulated carrier with 
average power equal to A^ll. The average noise power at the detector input is 

ai, = 2WN 0 (2.103) 

The carrier-to-noiser ratio is therefore defined by 

All! 

(2.104) 

4WN 0 

We may think of p as an input signal-to-noise ratio for the problem described herein. 



*A reader who is not interested in the mathematical details of how noise affects the envelope detection of an 
AM signal may skip the material up to Eq. (2,124) and read the two limiting cases of the formula in that equation. 
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However, determination of the output signal-to-noise ratio is a more difficult un- 
dertaking because the envelope detector Output 

y{t) = V(A C +' n,(t)) 2 + n%(t) (2.105) 

is a nonlinear combination of signal and noise terms. With no clear-cut separation between 
signal and noise at the detector output y(t), how then do we isolate the contribution of 
the signal s(f ) to y(t) from the contribution due to the noise «(f )? To resolve this issue, we 
adopt a heuristic approach based on signal averaging. Specifically, we introduce the fol- 
lowing two definitions: 

1. The mean output signal, s a , is the difference between the expectation of y{t) in the 
combined presence of signal and noise and the expectation of y{t) in the presence of 
noise alone, as shown by 

s 0 = E[y(t)] - E[y 0 (t)l (2.106) 
where y(t) is itself defined by Equation (2.105) and y 0 (t) is defined by 



y a {t) = Vnj(t) + n 2 B (t) (2.107) 

2. The mean output noise power is the difference between the mean-square value of the 
detector output y(t) and the square of the mean value of y{t), as shown by 

varbW] = E\y\t)\ - (E[y(t)}f (2.108) 
On this basis, we define the output signal-to-noise ratio as 

(SNR) 0 = *° (2.109) 
var[y(f)] 

From Section 1.12, we recall that the envelope detector output due to noise alone is 
Rayleigh distributed; that is 



i^Vh)' (2.110) 
.0 otherwise 



The expectation of y 0 (t) is therefore 



E\y 0 (t)\ = yfyjy) dy 



,2 ex P -T~i\dy 



o ai, \ 2ai, 

From the definition of the gamma function for real positive values of the argument x, we 
have 

V(x) = j o z'- 1 exp(-z) dz (2.112) 
We may therefore rewrite Equation (2.111) as 

EMt)] = V2<7«r(| 

(2.113) 

ITT 

hr cr N 
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where we have used the value T(3/2) = Vtt/2. To calculate the mean signal s„ at the 
detector output, we also need the expectation of y(t). Due to the combined presence of 
signal and noise, we recall from Section 1.13 that y(t) is Rician distributed, as shown by 

0 otherwise 



My) = 



where J 0 (') is the modified Bessel function of the first kind of zero order (see Appendix 3). 
Hence, 

Putting A c y/oj< = u and recognizing that p = A 1 J2o-l l , we may recast this expectation in 
the form 

E[y(t)] = j^r 2 exp(-p) | o u 2 exp(-^j/ 0 («) du (2.116) 

The integral in Equation (2.116) can be written in a concise form by using confluent 
hypergeometric functions; see Appendix 4. In particular, using the integral representation 

[ «p(-fcV)I 0 <«) du = Ig^j^ ;1-L) ) (2.117) 

with m = 3, T(m/2) = Vtt/2 and f> 2 = l/4p, we may express the expectation of y{t) in 
terms of the confluent hypergeometric function jp 1 (3/2;l;p) as 

E[y(t)] = «r N exp^p^^;!^ (2.118) 

We may further simplify matters by using the following identity: 

exp(-«)( 1 P 1 (a;ft«)) = JAP ~ osfr-u) (2.119) 
and so finally express the expectation of y(t) in the concise form 

Eb(t)] = ^(iPi(~|;1;-p)) (2.120) 

Thus using Equations (2.113) and (2.120) in Equation (2.106) yields the mean output 
signal as 

= <r N (iF,(-f sl;-p) - l) (2.121) 

whose dependence on the standard deviation cr N of the noise «(f) is testimony to the 
intermingling of signal and noise at the detector output. 

Following a similar procedure, we may express the mean-square value of the detector 
output y(t) as 

= 2 f r 2 .,( 1 f 1 (-l;l;-p)) 
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Hence using Equations (2.120) and (2.122) in Equation (2.108) yields the mean output 
noise power as 

var[y(t)] = 2<x 2 J ( 1 F 1 (-l;l;-p) - J^-I-i.-^ j (2 ,l 2 3) 

Finally, using Equations (2.121) and (2.123) in Equation (2.109) yields the output 
signal-to-noise ratio for the envelope detection problem at hand as 



(SNR)o = IK,,, ^ P-124) 

(4/ir}(A(-lil;-p)J - \iFi{~;1;-p 

Equation (2.124) is the general formula for the output signal-to-noise of an envelope 
detector whose input consists of an unmodulated carrier and band-limited, white Gaussian 
noise. Two limiting cases of this general formula are of particular interest: 

1. Large carrier-to-noise ratio. For large p, we may use the following asymptotic for- 
mula (see Appendix 4) 

lFl {~h 1 ''~ P ) " 2 J^r for p ~* °° (2 ' 125) 

Moreover, the following identity 

jfM-l^-p) = 1 + p (2.126) 

holds exactly for all p. Accordingly, the use of Equations (2.125) and (2.126) in 
Equation (2.124) yields the following approximate formula for the output signal-to- 
noise ratio: 

(SNR)o = p for p -» a- (2.127) 

where we have ignored contributions due to p 1 ' 2 and p° in the numerator of Equation 
(2.124) as being subdominant compared to p for large p. Equation (2.127) shows 
that for large carrier-to-noise p the envelope detector behaves like a coherent detector, 
in that the output signal-to-noise ratio is proportional to the input signal-to-noise 
ratio. 

2. Small carrier-to-noise ratio. For small p, we have (see Appendix 4) 

1 F 1 (a;c;-p) p for p ^ 0 (2.128) 

Hence, using this asymptotic formula, we may approximate the output signal-to- 
noise ratio for small p as 

(SNR) C 



16 4tt (2.129) 
=- 0.91p 2 for p -» 0 

where, in the denominator, we have ignored contributions due to p and p 2 as being 
subdominant compared to p° for small p. Equation (2.129) shows that for a small 
carrier-to-noise ratio, the output signal-to-noise ratio of the envelope detector is pro- 
portional to the squared input signal-to-noise ratio. 
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i.o 



Carrier-to-noise ratio, p 

Figure 2.39 Output signal-to-noise ratio of an envelope detector for varying carrier-to-noise 
ratio. 



The conclusions drawn from the two limiting cases considered herein are that an 
envelope detector favors strong signals and penalizes weak signals. The phenomenon of 
weak signals being penalized by the detector is referred to as weak signal suppression, 
which is a manifestation of the threshold effect. 

Using the formula of Equation (2.124), in Figure 2.39 we have plotted the output 
signal-to-noise ratio (SNR) G of the envelope detector versus the carrier-to-noise ratio p 
using tabulated values of confluent hypergeometric functions. This figure also includes the 
two asymptotes for large p and small p. From Figure 2.39 we see that the output signal- 
to-noise ratio deviates from a linear behavior around a carrier-to-noise ratio of 10 dB (i.e., 
p-10). 



2.13 Noise in FM Receivers 

Finally, we turn our attention to the noise analysis of a frequency modulation (FM) system, 
for which we use the receiver model shown in Figure 2.40. As before, the noise w(t) is 
modeled as white Gaussian noise of zero mean and power spectral density N 0 /2. The 
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FIGURE 2.40 Model of an FM receiver. 
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received FM signal s(t ) has a carrier frequency f c and transmission bandwidth B T , such 
that only a negligible amount of power lies outside the frequency band f c ± Bt/2 for 
positive frequencies, and similarly for negative frequencies. 

As in the AM case, the band-pass filter has a midband frequency f c and bandwidth 
B T and therefore passes the FM signal essentially without distortion. Ordinarily, B T is 
small compared with the midband frequency f c , so that we may use the narrowband 
representation for n(t), the filtered version of channel noise w(t), in terms of its in-phase 
and quadrature components. 

In an FM system, the message signal is transmitted by variations of the instantaneous 
frequency of a sinusoidal carrier wave, and its amplitude is maintained constant. Therefore, 
any variations of the carrier amplitude at the receiver input must result from noise or 
interference. The amplitude limiter, following the band-pass filter in the receiver model of 
Figure 2.40, is used to remove amplitude variations by clipping the modulated wave at 
the filter output almost to the zero axis. The resulting rectangular wave is rounded off by 
another band-pass filter that is an integral part of the limiter, thereby suppressing har- 
monics of the carrier frequency. Thus, the filter output is again sinusoidal, with an am- 
plitude that is practically independent of the carrier amplitude at the receiver input. 

The discriminator in the model of Figure 2.40 consists of two components: 

1. A slope network or differentiator with a purely imaginary frequency response that 
varies linearly with frequency. It produces a hybrid-modulated wave in which both 
amplitude and frequency vary in accordance with the message signal. 

2. An envelope detector that recovers the amplitude variation and thus reproduces the 
message signal. 

The slope network and envelope detector are usually implemented as integral parts of a 
single physical unit. 

The postdetection filter, labeled "baseband low-pass filter" in Figure 2.40, has a 
bandwidth that is just large enough to accommodate the highest frequency component of 
the message signal. This filter removes the out-of-band components of the noise at the 
discriminator output and thereby keeps the effect of the output noise to a minimum. 

The filtered noise n(t) at the band-pass filter output in Figure 2.40 is defined in terms 
of its in-phase and quadrature components by 



The envelope r(t) is Rayleigh distributed, and the phase ifr(t) is uniformly distributed over 
2tt radians (see Section 1.12). 

The incoming FM signal s(t) is defined by 




(2.131) 



(2.130) 



i/f(£) = tan 



i n uit) 



(2.132) 



nj(t) 




(2.133) 



Chapter 2 a Continuous-Wave Modulation 



where A c is the carrier amplitude, f c is the carrier frequency, k f is the frequency sensitivity, 
and m(t) is the message signal. Note that, as with the standard AM, in FM there is no 
need to introduce a scaling factor in the definition of the modulated signal s(t), since it i s 
reasonable to assume that its amplitude A c has the same units as the additive noise com- 
ponent n(t). To proceed, we define 



4>(f) = 217*/ jm(T) dr (2.134) 

We may thus express s(t) in the simple form 

s(f) = A c cos[2ir/ e f + 4>(f)] (2.135) 

The noisy signal at the band-pass filter output is therefore 

x(t) = s[t) + n(t) 

= A c cos\2irf c t + <t>[t)} + r(t) cos[2ir/ c t + i/r(f)] 

It is informative to represent x(t) by means of a phasor diagram, as in Figure 2.41. In this 
diagram we have used the signal term as reference. The phase 0(f) of the resultant phasor 
representing x(t) is obtained directly from Figure 2.41 as 

8(t) = 4>(t) + tan M — T7-7f (2.137) 

[A c + r(t) cos[i/<(f) - <l>(t)]) 

The envelope of x(t ) is of no interest to us, because any envelope variations at the band- 
pass filter output are removed by the limiter. 

Our motivation is to determine the error in the instantaneous frequency of the carrier 
wave caused by the presence of the filtered noise n(t). With the discriminator assumed 
ideal, its output is proportional to e'(t)/2ir where 0'(f) is the derivative of 0(f) with respect 
to time. In view of the complexity of the expression defining 0(f), however, we need to 
make certain simplifying approximations, so that our analysis may yield useful results. 

We assume that the carrier-to-noise ratio measured at the discriminator input is large 
compared with unity. Let R denote the random variable obtained by observing (at some 
fixed time) the envelope process with sample function r(t) [due to the noise n(t )]. Then, at 
least most of the time, the random variable R is small compared with the carrier amplitude 
A c , and so the expression for the phase 0(f) simplifies considerably as follows: 

0(f) - <Mf) + ^ sinWKf) - <Mt)] (2.1 38 ) 

A c 

or, using the expression for <p(t) given in Equation (2.134), 

0(f) - 271*/ f m(r) dr+ r -^ sin^ff) - <*>(*)] (2-139) 

Jo A c 




FIGURE 2.41 Phasor diagram for FM wave plus narrowband noise for the case of high carrier- 
to-noise ratio. 
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The discriminator output is therefore 

v(t) = 

where the noise term n d (t) is defined by 



1 d6(t) 

2 ^ dt (2.140) 
k f m{t) + n d (t) 



1 d 

"At) = j f {r(t) sinMt) - <Ht)]) (2.141) 

We thus see that provided the carrier-to-noise ratio is high, the discriminator output v(t) 
consists of the original message signal m(t) multiplied by the constant factor k f , plus an 
additive noise component n d [t). Accordingly, we may use the output signal-to-noise ratio 
as previously defined to assess the quality of performance of the FM receiver. Before doing 
this, however, it is instructive to see if we can simplify the expression defining the noise 
nAt). 

From the phasor diagram of Figure 2.41, we note that the effect of variations in the 
phase \j/(t) of the narrowband noise appear referred to the signal term c/> (f ). We know that 
the phase ij/(t) is uniformly distributed over 2tt radians. It would therefore be tempting to 
assume that the phase difference if>(t) - <f> (t) is also uniformly distributed over 2tt radians. 
If such an assumption were true, then the noise n d (t) at the discriminator output would 
be independent of the modulating signal and would depend only on the characteristics of 
the carrier and narrowband noise. Theoretical considerations show that this assumption 
is justified provided that the carrier-to-noise ratio is high. 9 Then we may simplify Equation 
(2.141) as: 

1 d 

"At) - j^- j f W) sinMf)]} (2.142) 

However, from the defining equations for r{t) and \p(t), we note that the quadrature com- 
ponent n Q (t) of the filtered noise n(t) is 

n Q (t) = r(t) sinMf)] (2.143) 

Therefore, we may rewrite Equation (2.142) as 

1 dn 0 (t) 

n ^-^dT < 2 - 144 > 

This means that the additive noise n d (t) appearing at the discriminator output is deter- 
mined effectively by the carrier amplitude A c and the quadrature component n Q (t) of the 
narrowband noise n(t). 

The output signal-to-noise ratio is defined as the ratio of the average output signal 
power to the average output noise power. From Equation (2.140), we see that the message 
component in the discriminator output, and therefore the low-pass filter output, is k f m(t). 
Hence, the average output signal power is equal to k}P, where P is the average power of 
the message signal m(t). 

To determine the average output noise power, we note that the noise n d (t) at the 
discriminator output is proportional to the time derivative of the quadrature noise com- 
ponent n Q (t). Since the differentiation of a function with respect to time corresponds to 
multiplication of its Fourier transform by jlirf, it follows that we may obtain the noise 
process n d (t) by passing n Q (t) through a linear filter with a frequency response equal to 

= jf_ 
2ttA c A c 
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This means that the power spectral density S Nd (f) of the noise n d (t) is related to the power 
spectral density S NQ {f) of the quadrature noise component n Q (t) as follows: 

With the band-pass filter in the receiver model of Figure 2.40 having an ideal fre- 
quency response characterized by bandwidth B T and midband frequency f c , it follows that 
the narrowband noise «(t) will have a power spectral density characteristic that is similarly 
shaped. This means that the quadrature component n a (t) of the narrowband noise n(t) 
will have the ideal low-pass characteristic shown in Figure 2.42a. The corresponding power 
spectral density of the noise n d (t) is shown in Figure 2.42&; that is, 



S N J,f) = \~Af' 1/1 ~2 (2-146) 




In the receiver model of Figure 2.40, the discriminator output is followed by a low-pass 
filter with a bandwidth equal to the message bandwidth W. For wideband FM, we usually 
find that W is smaller than B T I2, where B T is the transmission bandwidth of the FM 
signal/This means that the out-of-band components of noise n d (t) will be rejected. There- 
fore, the power spectral density S No (f) of the noise n 0 (t) appearing at the receiver output 
is defined by 




\f\^W (2.147) 
otherwise 



as shown in Figure 2.42c. The average output noise power is determined by integrating 
the power spectral density S No (f) from - W to W. We thus get the following result: 

N f w 

Average power of output noise = f 2 df 

A < (2.148) 
_ 2N 0 W 3 

~ 3A 2 C 
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Figure 2.42 Noise analysis of FM receiver, (a) Power spectral density of quadrature compo- 
nent » e (t) of narrowband noise n(t). (b) Power spectral density of noise «,,(t) at the discriminator 
output, (c) Power spectral density of noise n 0 (t) at the receiver output. 
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Note that the average output noise power is inversely proportional to the average carrier 
power A 2 /2. Accordingly, in an FM system, increasing the carrier power has a noise- 
quieting effect. 

Earlier we determined the average output signal power as kjP. Therefore, provided 
the carrier-to-noise ratio is high, we may divide this average output signal power by the 
average output noise power of Equation (2.148) to obtain the output signal-to-noise ratio 

3A 2 k 2 P 

(SNR)o '™ = 2N~^ < 2 - 149 > 

The average power in the modulated signal s(t) is A 2 J2, and the average noise power in 
the message bandwidth is WN 0 . Thus the channel signal-to-noise ratio is 

(SNR)c ' FM = 2§N 0 (Z150 > 

Dividing the output signal-to-noise ratio by the channel signal-to-noise ratio, we get the 
following figure of merit for frequency modulation: 



(SNR) 0 



(SNR) C 



3k}P 

-^T (2.151) 



From Section 2.7 we recall that the frequency deviation Af is proportional to the 
frequency sensitivity k f of the modulator. Also, by definition, the deviation ratio D is equal 
to the frequency deviation Af divided by the message bandwidth W. In other words, the 
deviation ratio D is proportional to the ratio k f P y2 /W. It follows therefore from Equation 
(2.151) that the figure of merit of a wideband FM system is a quadratic function of the 
deviation ratio. Now, in wideband FM, the transmission bandwidth B T is approximately 
proportional to the deviation ratio D. Accordingly, we may state that when the carrier- 
to-noise ratio is high, an increase in the transmission bandwidth B T provides a correspond- 
ing quadratic increase in the output signal-to-noise ratio or figure of merit of the FM 
system.The important point to note from this statement is that, unlike amplitude modu- 
lation, the use of frequency modulation does provide a practical mechanism for the ex- 
change of increased transmission bandwidth for improved noise performance. 



► Example 2.5 Single-Tone Modulation 

Consider the case of a sinusoidal wave of frequency f m as the modulating signal, and assume 
a peak frequency deviation A/. The modulated FM signal is thus defined by 



s(t) = A c cos 



+ j- sin(2ir/ m f) 



Therefore, we may write 



2-rrk 



■f Mt) 
Jo 



A/" 

dr = -j- sin(2irf„t) 



Differentiating both sides with respect to time and solving for m(t), we get 

A/ 

m(f) = -j- cos(2ir/„rt 
*/ 

Hence, the average power of the message signal m(t), developed across a 1-ohm load, i 
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Substituting this result into the formula for the output signal-to-noise ratio given in Equation 
(2.149), we get 

(SNR)o,fm = ■ 



4N 0 W 3 
3Ajp 2 
' 4N 0 W 



where /3 = A// Wis the modulation index. Using Equation (2.151) to evaluate the correspond- 
ing figure of merit, we get 



(SNR) 0 



(SNR)c 



(2.152) 



FM 



It is important to note that the modulation index 0 = A/AW is determined by the bandwidth 
W of the postdetection low-pass filter and is not related to the sinusoidal message frequency 
f„, except insofar as this filter is usually chosen so as to pass the spectrum of the desired 
message; this is merely a matter of consistent design. For a specified system bandwidth W, the 
sinusoidal message frequency /„ may lie anywhere between 0 and W and would yield the same 
output signal-to-noise ratio. 

It is of particular interest to compare the noise performance of AM and FM systems. 
An insightful way of making this comparison is to consider the figures of merit of the two 
systems based on a sinusoidal modulating signal. For an AM system operating with a sinu- 
soidal modulating signal and 100 percent modulation, we have (from Example 2.4): 



(SNR)o 



(SNR) C 



Comparing this figure of merit with the corresponding result described in Equation (2.152] 
for an FM system, we see that the use of frequency modulation offers the possibility of im- 
proved noise performance over amplitude modulation when 



that is, 



V2 

: > — = 0.471 
3 



We may therefore consider f} = 0.5 as defining roughly the transition between narrowband 
FM and wideband FM. This statement, based on noise considerations, further confirms 
a similar observation that was made in Section 2.7 when considering the bandwidth of 
FM waves. ^ 



■ Capture Effect 

The inherent ability of an FM system to minimize the effects of unwanted signals (e.g., 
noise, as just discussed) also applies to interference produced by another frequency- 
modulated signal whose frequency content is close to the carrier frequency of the desired 
FM wave. However, interference suppression in an FM receiver works well only when the 
interference is weaker than the desired FM input. When the interference is the strong^ 
one of the two, the receiver locks onto the stronger signal and thereby suppresses the 
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desired FM input. When they are of nearly equal strength, the receiver fluctuates back and 
forth between them. This phenomenon is known as the capture effect, which describes 
another distinctive characteristic of frequency modulation. 

■ FM Threshold Effect 

The formula of Equation (2.149), defining the output signal-to-noise ratio of an FM re- 
ceiver, is valid only if the carrier-to-noise ratio, measured at the discriminator input, is 
high compared with unity. It is found experimentally that as the input noise power is 
increased so that the carrier-to-noise ratio is decreased, the FM receiver breaks. At first, 
individual clicks are heard in the receiver output, and as the carrier-to-noise ratio decreases 
still further, the clicks rapidly merge into a crackling or sputtering sound. Near the break- 
ing point, Equation (2.149) begins to fail by predicting values of output signal-to-noise 
ratio larger than the actual ones. This phenomenon is known as the threshold effect} 0 The 
threshold is defined as the minimum carrier-to-noise ratio yielding an FM improvement 
that is not significantly deteriorated from the value predicted by the usual signal-to-noise 
formula assuming a small noise power. 

For a qualitative discussion of the FM threshold effect, consider first the case when 
there is a no signal present, so that the carrier wave is unmodulated. Then the composite 
signal at the frequency discriminator input is 



where n x (f) and n Q (t) are the in-phase and quadrature components of the narrowband 
noise n(t) with respect to the carrier wave. The phasor diagram of Figure 2.43 displays 
the phase relations between the various components of x(t) in Equation (2.153). As the 
amplitudes and phases of «j(f) and n Q (t) change with time in a random manner, the point 
Pi [the tip of the phasor representing x(t)} wanders around the point P z (the tip of the 
phasor representing the carrier). When the carrier-to-noise ratio is large, Kj(f) and n Q (t) 
are usually much smaller than the carrier amplitude A c , and so the wandering point P, in 
Figure 2.43 spends most of its time near point P 2 . Thus the angle 0(f) is approximately 
n Q (t)/A c to within a multiple of 2ir. When the carrier-to-noise ratio is low, on the other 
hand, the wandering point P x occasionally sweeps around the origin and 0(f) increases or 
decreases by 2tt radians. Figure 2.44 illustrates how in a rough way the excursions in 0(f), 
depicted in Figure 2.44a, produce impulselike components in 0'(f) = dd/dt. The discrim- 
inator output v(t) is equal to 0'(f )/2ir. These impulselike components have different heights 
depending on how close the wandering point P 1 comes to the origin O, but all have areas 
nearly equal to ±2ir radians, as illustrated in Figure 2.44b. When the signal shown in 
Figure 2.44b is passed through the postdetection low-pass filter, corresponding but wider 
impulselike components are excited in the receiver output and are heard as clicks. The 
clicks are produced only when 0(f) changes by ±2ir radians. 



x(t) = [A c + n r {t)] cos(27T/ c f) - n Q (t) sm(2irf c t) 



(2.153) 




Figure 2.43 Phasor diagram interpretation of Equation (2. 1 53). 
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Figure 2.44 Illustrating impulselike components in 6'l.t) = de(t)/d* produced by changes of 2z 
in 0(t); («) and (b) are graphs of 8(t) and fl'(t), respectively. 



From the phasor diagram of Figure 2.43, we may deduce the conditions required for 
clicks to occur. A positive-going click occurs when the envelope r(t) and phase ^(f) of the 
narrowband noise n(t) satisfy the following conditions: 

r(t) > A c 
,/,({) < it < i/-(f) + d<l>(t) 

#(£) >0 
dt 

These conditions ensure that the phase 0(t) of the resultant phasor x(t) changes by 2ir 
radians in the time increment dt, during which the phase of the narrowband noise increases 
by incremental amount d^(t)- Similarly, the conditions for a negative-going click to occur 
are as follows: 

r(t) > A c 
</»({) > -77 > ip(t) + dip(t) 

dm <0 

dt 

These conditions ensure that 9(f) changes by -2ir radians during the time increment dt. 
The carrier-to-noise ratio is defined by 

(2.154) 



F 2B T N 0 

As p is decreased, the average number of clicks per unit time increases. When this number 
becomes appreciably large, threshold is said to occur. 
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The output signal-to-noise ratio is calculated as follows: 

1. The output signal is taken as the receiver output measured in the absence of noise. 
The average output signal power is calculated assuming a sinusoidal modulation that 
produces a frequency deviation A/ equal to so that the carrier swings back 
and forth across the entire input frequency band. 

2. The average output noise power is calculated when there is no signal present; that 
is, the carrier is unmodulated, with no restriction imposed on the value of the carrier- 
to-noise ratio p. 

On this heuristic basis, theory 11 yields Curve I of Figure 2.45 presenting a plot of the 
output signal-to-noise ratio versus the carrier-to-noise ratio when the ratio B T /2 Wis equal 
to 5. This curve shows that the output signal-to-noise ratio deviates appreciably from a 
linear function of the carrier-to-noise ratio p when p is less than about 10 dB. Curve II of 
Figure 2.45 shows the effect of modulation on the output signal-to-noise ratio when the 
modulating signal (assumed sinusoidal) and the noise are present at the same time. The 
average output signal power pertaining to curve II may be taken to be effectively the same 
as for curve I. The average output noise power, however, is strongly dependent on the 
presence of the modulating signal, which accounts for the noticeable deviation of curve II 
from curve I. In particular, we find that as p decreases from infinity, the output signal-to- 




Carrier-to-noise ratio 10]og 10 p, dB 

Figure 2.45 Dependence of output signal-to-noise ratio on input carrier-to-noise ratio for FM 
reciever. In curve I, the average output noise power is calculated assuming an unmodulated car- 
rier. In curve II, the average output noise power is calculated assuming a sinusoidally modulated 
carrier. Both curves I and II are calculated from theory. 
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noise deviates appreciably from a linear function of p when p is about 11 dB. Also when 
the signal is present, the resulting modulation of the carrier tends to increase the avera ge 
number of clicks per second. Experimentally, it is found that occasional clicks are heard 
in the receiver output at a carrier-to-noise ratio of about 13 dB, which appears to be only 
slightly higher than what theory indicates. Also it is of interest to note that the increase in , 
the average number of clicks per second tends to cause the output signal-to-noise ratio to 
fall off somewhat more sharply just below the threshold level in the presence of 
modulation. 

From the foregoing discussion we may conclude that threshold effects in FM receivers 
may be avoided in most practical cases of interest if the carrier-to-noise ratio p is equal to 
or greater than 20 or, equivalently, 13 dB. Thus using Equation (2.154) we find that the 
loss of message at the discriminator output is negligible if 

A* 

■ > 20 



2B T N 0 

or, equivalently, if the average transmitted power A 2 J2 satisfies the condition 

^ a 20B T N 0 (2.155): 

To use this formula, we may proceed as follows: 

1. For a specified modulation index /3 and message bandwidth W, we determine the 
transmission bandwidth of the FM wave, B T , using the universal curve of Figure 2,26 
or Carson's rule. 

2. For a specified average noise power per unit bandwidth, N 0 , we use Equation (2.155) 
to determine the minimum value of the average transmitted power A\I2 that is nec- 
essary to operate above threshold. 

■ FM Threshold Reduction 

In communication systems using frequency modulation, there is particular interest in re- 
ducing the noise threshold in an FM receiver so as to satisfactorily operate the receiver 
with the minimum signal power possible. Threshold reduction in FM receivers may be 
achieved by using an FM demodulator with negative feedback 12 (commonly referred to as 
an FMFB demodulator), or by using a phase-locked loop demodulator. Such devices are 
referred to as extended-threshold demodulators, the idea of which is illustrated in Figure 
2.46. The threshold extension shown in this figure is measured with respect to the standard 
frequency discriminator (i.e., one without feedback). 

The block diagram of an FMFB demodulator 13 is shown in Figure 2.47. We see that 
the local oscillator of the conventional FM receiver has been replaced by a voltage- 
controlled oscillator (VCO) whose instantaneous output frequency is controlled by the 
demodulated signal. In order to understand the operation of this receiver, suppose for the 
moment that the VCO is removed from the circuit and the feedback loop is left open. 
Assume that a wideband FM signal is applied to the receiver input, and a second FM 
signal, from the same source but whose modulation index is a fraction smaller, is applied 
to the VCO terminal of the mixer. The output of the mixer would consist of the difference 
frequency component, because the sum frequency component is removed by the band-pass 
filter. The frequency deviation of the mixer output would be small, although the frequency 
deviation of both input FM waves is large, since the difference between their instantaneous 
deviations is small. Hence, the modulation indices would subtract and the resulting FM 
wave at the mixer output would have a smaller modulation index. The FM wave with 




reduced modulation index may be passed through a band-pass filter, whose bandwidth 
need only be a fraction of that required for either wideband FM, and then frequency 
demodulated. It is now apparent that the second wideband FM signal applied to the mixer 
may be obtained by feeding the output of the frequency discriminator back to the VCO. 

It will now be argued that the signal-to-noise ratio of an FMFB receiver is the same 
as that of a conventional FM receiver with the same input signal and noise power if the 
carrier-to-noise ratio is sufficiently large. Assume for the moment that there is no feed- 
back around the demodulator. In the combined presence of an unmodulated carrier 
A c cos(2tt/ c £) and narrowband noise 

n(t) = n T (t) cos(2i7-£f) - n Q (t) sin(2nfj) 

the phase of the composite signal x(t) at the limiter-discriminator input is approximately 
equal to n Q (t)IA c , assuming that the carrier-to-noise ratio is high. The envelope of x(t) is 
of no interest to us, because the limiter removes all variations in the envelope. Thus the 
composite signal at the frequency discriminator input consists of a small index phase- 
modulated wave with the modulation derived from the component n B (t) of noise that is 
in phase quadrature with the carrier. When feedback is applied, the VCO generates a 
frequency-modulated signal that reduces the phase-modulation index of the wave in the 
band-pass filter output, that is, the quadrature component rag(f) of noise. Thus we see that 
as long as the carrier-to-noise ratio is sufficiently large, the FMFB receiver does not respond 
to the in-phase noise component n T (t), but that it would demodulate the quadrature noise 
component n Q (t) in exactly the same fashion as it would demodulate signal modulation. 
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Figure 2.47 FM demodulator with negative feedback. 
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Signal and quadrature noise are reduced in the same proportion by the applied feedback, 
with the result that the baseband signal-to-noise ratio is independent of feedback. For large 
carrier-to-noise ratios, the baseband signal-to-noise ratio of an FMFB receiver is then the 
same as that of a conventional FM receiver. 

The reason that an FMFB receiver is able to extend the threshold is that, unlike a 
conventional FM receiver, it uses a very important piece of a priori information, namely, 
that even though the carrier frequency of the incoming FM wave will usually have large 
frequency deviations, its rate of change will be at the baseband rate. An FMFB demodu- 
lator is essentially a tracking filter that can track only the slowly varying frequency of a 
wideband FM signal, and consequently it responds only to a narrowband of noise centered 
about the instantaneous carrier frequency. The bandwidth of noise to which the FMFB 
receiver responds is precisely the band of noise that the VCO tracks. The end result is that 
an FMFB receiver is capable of realizing a threshold extension on the order of 5-7 dB, 
which represents a significant improvement in the design of minimum power FM systems. 

Like the FMFB demodulator, the phase-locked loop (discussed later in Section 2.14) 
is also a tracking filter and, as such, the noise bandwidth to which it responds is precisely 
the band of noise tracked by the VCO. Indeed, the phase-locked loop demodulator offers 
a threshold extension capability with a relatively simple circuit. Unfortunately, the amount 
of threshold extension is not predictable by any existing theory, and it depends on signal 
parameters. Roughly speaking, improvement by a few (on the order of 2 to 3) decibels is 
achieved in typical applications, which is not as good as an FMFB demodulator. 



■ Pre-Emphasis and De-Emphasis in FM 

Equation (2.147) shows that the power spectral density of the noise at the output of an 
FM receiver has a square-law dependence on the operating frequency; this is illustrated in 
Figure 2.48«, In Figure 2.48b, we have included the power spectral density of a typical 
message source; audio and video signals typically have spectra of this form. In particular, 
we see that the power spectral density of the message usually falls off appreciably at higher 
frequencies. On the other hand, the power spectral density of the output noise increases 
rapidly with frequency. Thus around / = ± W, the relative spectral density of the message 
is quite low, whereas that of the output noise is quite high in comparison. Clearly, the 
message is not using the frequency band allotted to it in an efficient manner. It may appeal 
that one way of improving the noise performance of the system is to slightly reduce the 
bandwidth of the postdetection low-pass filter so as to reject a large amount of noise power 
while losing only a small amount of message power. Such an approach, however, is usually 
not satisfactory because the distortion of the message caused by the reduced filter band- 
width, even though slight, may not be tolerable. For example, in the case of music, we 
find that although the high-frequency notes contribute only a very small fraction of the 
total power, nonetheless, they contribute a great deal from an esthetic viewpoint. 

A more satisfactory approach to the efficient use of the allowed frequency band is 
based on the use of pre-emphasis in the transmitter and de-emphasis in the receiver, as 



■V</1 
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Figure 2.48 (a) Power spectral density of noise at FM receiver output, (h) Power spectral den- 
sity of a typical message signal. 



2. 1 3 Noise in FM Receivers 155 



Pre-emphasis 




FM 


filter, HJf) 




transmitter 




FM 




De-emphasis 


receiver 




filter, HJf) 



Message 
plus noise 



Noise 
k ft) 



Figure 2.49 Use of pre-emphasis and de-emphasis in an FM system. 



illustrated in Figure 2.49. In this method, we artificially emphasize the high-frequency 
components of the message signal prior to modulation in the transmitter, and therefore 
before the noise is introduced in the receiver. In effect, the low-frequency and high- 
frequency portions of the power spectral density of the message are equalized in such a 
way that the message fully occupies the frequency band allotted to it. Then, at the discrim- 
inator output in the receiver, we perform the inverse operation by de-emphasizing the 
high-frequency components, so as to restore the original signal-power distribution of the 
message. In such a process, the high-frequency components of the noise at the discriminator 
output are also reduced, thereby effectively increasing the output signal-to-noise ratio of 
the system. Such a pre-emphasis and de-emphasis process is widely used in commercial 
FM radio transmission and reception. 

In order to produce an undistorted version of the original message at the receiver 
output, the pre-emphasis filter in the transmitter and the de-emphasis filter in the receiver 
must ideally have frequency responses that are the inverse of each other. That is, if H pe (/) 
designates the frequency response of the pre-emphasis filter, then the frequency response 
Hie(f) °f the de-emphasis filter must ideally be (ignoring transmission delay) 



H de (/) = 



1 



H pe (/)' 



(2.156) 



This choice of frequency responses makes the average message power at the receiver output 
independent of the pre-emphasis and de-emphasis procedure. 

From our previous noise analysis in FM systems, assuming a high carrier-to-noise 
ratio, the power spectral density of the noise n d (t) at the discriminator output is given by 
Equation (2.146). The modified power spectral density of the noise at the de-emphasis 
filter output is therefore 



l«d«(/)| 2 5 N ,(/) 



|H de (/)| 



otherwise 



(2.157) 



Recognizing, as before, that the postdetection low-pass filter has a bandwidth W that is, 
in general, less than B T /2, we find that the average power of the modified noise at the 
receiver output is as follows: 



Average output noise 
power with de-emphasis 



No f 
' A 2 J- 



(2.158) 



Because the average message power at the receiver output is ideally unaffected by the 
combined pre-emphasis and de-emphasis procedure, it follows that the improvement in 
output signal-to-noise ratio produced by the use of pre-emphasis in the transmitter and 
de-emphasis in the receiver is defined by 



average output noise power without pre-emphasis and de-emphasis 
average output noise power with pre-emphasis and de-emphasis 



(2.159) 
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Earlier we showed that the average output noise power without pre-emphasis and de- 
emphasis is equal to (2N 0 W 3 /3A 2 ); see Equation (2.148). Therefore, after cancellation of 
common terms, we may express the improvement factor J as 

2W3 

I = (2-160) 

3 J_ w / 2 |H de (/)| 2 J/ 

It must be emphasized that this improvement factor assumes the use of a high carrier-to 
noise ratio at the discriminator input in the receiver. 



> Example 2.6 

A simple pre-emphasis filter that emphasizes high frequencies and is commonly used in practice 
is denned by the frequency response 

Hpo(/) = 1 + 7 
Jo 

which is closely realized by the RC-amplifier network shown in Figure 2.50<J, provided that 
R « r and lirfCr « 1 inside the frequency band of interest. The amplifier in Figure 2.50a 
is intended to make up for the attenuation introduced by the RC network at low frequencies. 
The frequency parameter f 0 is l/(2irCr). 

The corresponding de-emphasis filter in the receiver is defined by the frequency response 

Hde(/) ■ 



1 + /Wo 

which can be realized using the simple RC network of Figure 2.50b. 

The improvement in output signal-to-noise ratio of the FM receiver, resulting from the 
combined use of the pre-emphasis and de-emphasis filters of Figure 2.50, is therefore 

_ _ 2W 3 



W fdf 

-w 1 + (f/fof (2.161) 
(W// 0 ) 3 



3[(W// 0 ) - tan'^W/o)] 



In commercial FM broadcasting, we typically have f 0 = 2.1 kHz, and we may reason- 
ably assume W = 15 kHz. This set of values yields I = 22, which corresponds to an improve- 
ment of 13 dB in the output signal-to-noise ratio of the receiver. The output signal-to-noise 
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Figure 2.50 (a) Pre-emphasis filter, (b) De-emphasis filter. 
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ratio of an FM receiver without pre-emphasis and de-emphasis is typically 40-50 dB. We see, 
therefore, that by using the simple pre-emphasis and de-emphasis filters shown in Figure 2.50, 
we can realize a significant improvement in the noise performance of the receiver. -M 

The use of the simple linear pre-emphasis and de-emphasis filters just described is an 
example of how the performance of an FM system may be improved by using the differ- 
ences between characteristics of signals and noise in the system. These simple filters also 
find application in audio tape-recording. Specifically, nonlinear pre-emphasis and de- 
emphasis techniques have been applied successfully to tape recording. These techniques 14 
(known as Dolby-A, Dolby-B, and DBX systems) use a combination of filtering and dy- 
namic range compression to reduce the effects of noise, particularly when the signal level 
is low. 



2.14 Computer Experiments: 
Phase-Locked Loop 

The experimental study presented in this section focuses on the use of a phase-locked loop 
for the demodulation of a frequency modulated signal. Before proceeding with the exper- 
iments, however, we first present a brief exposition of phase-locked loop theory. 

Basically, the phase-locked loop consists of three major components: a multiplier, a 
loop filter, and a voltage-controlled oscillator (VCO) connected together in the form of a 
feedback system, as shown in Figure 2.51. The VCO is a sinusoidal generator whose 
frequency is determined by a voltage applied to it from an external source. In effect, any 
frequency modulator may serve as a VCO. We assume that initially we have adjusted the 
VCO so that when the control voltage is zero, two conditions are satisfied: 

1. The frequency of the VCO is precisely set at the unmodulated carrier frequency f c . 

2. The VCO output has a 90 degree phase-shift with respect to the unmodulated carrier 
wave. 

Suppose then that the input signal applied to the phase-locked loop is an FM signal defined 
by 

s{t) = A c sin[2w/ c r + tp^t)] 

where A c is the carrier amplitude. With a modulating signal m(t), the angle tp^t) is related 
to m(t) by the integral 

$i(t) = lukf J" m(r) dr 
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FIGURE 2.51 Phase-locked loop. 
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where k f is the frequency sensitivity of the frequency modulator. Let the VCO output i n 
the phase-locked loop be denned by 

r(f) = A v cos[2ir/ c t + 4> 2 (t)} 
where A v is the amplitude. With a control voltage v(t) applied to the VCO input, the angle 
4>z(t) is related to v(t) by the integral 

<j> 2 (t) = lnk v J" o v(t) dr (2.162) 

where k v is the frequency sensitivity of the VCO, measured in Hertz per volt. The object 
of the phase-locked loop is to generate a VCO output r(f) that has the same phase angle 
(except for the fixed difference of 90 degrees) as the input FM signal s{t). The time-varying 
phase angle <£j(f) characterizing s(t) may be due to modulation by a message signal m[t), 
in which case we wish to recover and thereby produce an estimate of m(t). In other 
applications of the phase-locked loop, the time-varying phase angle <j> , (f ) of the incoming 
signal s(t) may be an unwanted phase shift caused by fluctuations in the communication 
channel; in this latter case, we wish to track <j> j(t ) so as to produce a signal with the same 
phase angle for the purpose of coherent detection (synchronous demodulation). 

■ Model of the Phase-Locked Loop 1 ' 

To develop an understanding of the phase-locked loop, it is desirable to have a model of 
the loop. We start by developing a nonlinear model, which is subsequently linearized to 
simplify the analysis. According to Figure 2.51, the incoming FM signal s(f) and the VCO 
output r(t) are applied to the multiplier, producing two components: 

1. A high-frequency component, represented by the double-frequency term 

kJLAv sin[47r/ 6 f + 0 i(f) + 4>i(t)] 

2. A low-frequency component represented by the difference-frequency term 

fe^AA.sint'Mf) - 0 2 (*)] 
where k„ is the multiplier gain, measured in volt -1 . 

The loop filter in the phase-locked loop is a low-pass filter, and its response to the high- 
frequency component will be negligible. The VCO also contributes to the attenuation of 
this component. Therefore, discarding the high-frequency component (i.e., the double- 
frequency term), the input to the loop filter is reduced to 

e(t) = k m A c A, sinrA(J)] (2.163) 
where <j> e (t) is the phase error defined by 

<Mt) = " <p2(t) (2.164) 

= 0!(f) - 2irk„ j" o v(r) dr 

The loop filter operates on the error e(t) to produce an output v(t) defined by the convo- 
lution integral: 

v(t) = [ e(r)h{t - t) dr (2.165! 
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where h(t) is the impulse response of the loop filter. Using Equations (2.164) and (2.165) 
to relate <f> e (t) and <f>i(t), we obtain the following nonlinear integro-differential equation 
as the descriptor of the dynamic behavior of the phase-locked loop: 



d<l>.(t) _ d<^ 1 (t) 



dt 



dt 



2ttK 0 sm[<t> e (T)]h(t - r) dr 



where K 0 is as loop-gain parameter defined by 



K 0 = k„k„A c A„ 



(2.166) 



(2.167) 



The amplitudes A c and A„ are both measured in volts, the multiplier gain k m in volt -1 and 
the frequency sensitivity k v in Hertz per volt. Hence, it follows from Equation (2.167) that 
K 0 has the dimensions of frequency. Equation (2.166) suggests the model shown in Figure 
2.52 for a phase-locked loop. In this model we have also included the relationship between 
v(t) and e(t) as represented by Equations (2.163) and (2.165). We see that the model of 
Figure 2.52 resembles the actual block diagram of Figure 2.51. The multiplier at the input 
of the phase-locked loop is replaced by a subtracter and a sinusoidal nonlinearity, and the 
VCO by an integrator. 

The sinusoidal nonlinearity in the model of Figure 2.52 complicates the task of an- 
alyzing the behavior of the phase-locked loop. It would be helpful to linearize this model 
to simplify the analysis and yet give a good approximate description of the loop's behavior 
in certain modes of operation. When the phase error <f>„(t) is zero, the phase-locked loop 
is said to be in phase-lock. When 4> e (t) is at all times small compared with one radian, we 
may use the approximation 

sin [&(*)] =* 4>„(t) 

which is accurate to within 4 percent for <f> e (t) less than 0.5 radians. In this case, the loop 
is said to be near phase-lock, and the sinusoidal nonlinearity of Figure 2.52 may be dis- 
regarded. Under this condition, v{t) is approximately equal to m(t), except for the scaling 
factor kflk v . 

The complexity of the phase-locked loop is determined by the frequency response 
H(f) of the loop filter. The simplest form of a phase-locked loop is obtained when 
H{f) = 1; that is, there is no loop filter, and the resulting phase-locked loop is referred to 
as a first-order phase-locked loop. A major limitation of a first-order phase-locked loop is 
that the loop gain parameter K 0 controls both the loop bandwidth as well as the hold-in 
frequency range of the loop; the hold-in frequency range refers to the range of frequencies 




Figure 2.52 Nonlinear model of the phase-locked loop. 
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for which the loop remains phase-locked to the input signal. We may overcome this lim- 
itation by using a loop filter with the frequency response 

H(/) = 1 + j (2.168) 

where a is a constant. Then with this loop filter in place and the phase-locked loop oper- 
ating in its linear mode, we find from Equation (2.166) that the phase-locked loop behaves 
as a second-order feedback system, as shown by the standard frequency response 

».(/) = WUf (2 l69i 

*!(/) 1 + 2 ((jf/Q + (iff*) 2 ' ' 

where *,(/) and *,(/) are the Fourier transforms of <f>.{t) and <M*)> respectively. The 
system is parameterized by the natural frequency, f„, and damping factor, {, which are 
respectively defined by 

f„ = VaK 0 (2.170) 

and 

f= P (2.171) 
y 4a 

The second-order phase-locked loop so described is the subject of the computer experi- 
ments presented next. 



Experiment 1: Acquisition Mode 

When a phase-locked loop is used for coherent detection (synchronous demodulation), the 
loop must first lock onto the input signal and then follow the variations of its phase angle 
with time. The process of bringing a loop into phase-lock is called acquisition, and the 
ensuing process of following angular variations in the input signal is called tracking. In 
the acquisition mode and quite possibly the tracking mode, the phase error <f> e (t) between 
the input signal s{t) and the VCO output r{t) will certainly be large, thereby mandating 
the use of the nonlinear model of Figure 2.52. However, a nonlinear analysis of the ac- 
quisition process based on this latter model is beyond the scope of this book. In this 
experiment, we use computer simulations to study the acquisition process and thereby 
develop insight into some of its features. 

Consider a second-order phase-locked loop using the loop filter of Equation (2.168) 
and having the following parameters: 

Natural frequency /„ = — Hz 
Lit 

Damping factor f = 0.3, 0.707, 1.0 
To accommodate variation in f, the filter parameter a is varied in accordance with the 
formula 

which follows from Equations (2.170) and (2.171). Figure 2.53 presents the variation m 
the phase error <j> e (t) with time for each of the three specified values of damping factor £ 
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Figure 2.53 Variation of the phase error for three different values of damping factor. 

assuming a frequency step of 0.125 Hz. These results show that the damping factor 
l = 0.707 gives the best compromise between a fast response time and an underdamped 
oscillatory behavior. 



Experiment 2: Phase-Plane Portrait 

A phase-plane portrait is a family of trajectories, with each trajectory representing a single 
solution of Equation (2.166). For the second experiment we plot the phase-plane portrait 
of a second-order phase-locked loop for the case of sinusoidal modulation. The system 
parameters of the loop are as follows: 

50 

Loop-gain parameter K 0 = — Hz 

Loop-natural frequency f„ = ^~7^ Hz 

Sinusoidal modulation frequency f m = — ~7= Hz 

2ctV2it 

Figure 2.54 presents the phase-plane portrait of the phase-locked loop adjusted for 
critical damping, where the trajectories (frequency error versus phase error) are plotted 
for different starting points. From this portrait we make the following observations: 

1. For a sinusoidal nonlinearity, the phase-plane portrait is itself periodic with period 
2ir in the phase error $ e , but it is aperiodic in d<j>Jdt. 

2. For an initial frequency error 

1 d<j> e 
K dt 

with an absolute value less than or equal to 1, the phase-locked loop is assured of 
attaining a stable (equilibrium) point at (0, 0) or (0, 2ir); the multiplicity of equilib- 
rium points is a manifestation of periodicity of the phase-plane portrait. 
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FIGURE 2.54 Phase-plane portrait for critical damping and sinusoidal modulation. 
3. For an initial frequency error 

ld& 
K dt 

with an absolute value equal to 2, we have a saddle point at (0, it) where the slightest 
perturbation applied to the phase-locked loop causes it to shift to the equilibrium 
point (0, 0) or (0, 2-ir). 

2.15 Summary and Discussion 

In this chapter we studied the principles of continuous-wave (CW) modulation. This an- 
alog form of modulation uses a sinusoidal carrier whose amplitude or angle is varied in 
accordance with a message signal. We may thus distinguish two families of CW modula- 
tion: amplitude modulation and angle modulation. 

■ Amplitude Modulation 

Amplitude modulation may itself be classified into four types, depending on the spectral 
content of the modulated signal. The four types of amplitude modulation and their prac- 
tical merits are as follows: 

1. Full amplitude modulation (AM), in which the upper and lower sidebands are trans- 
mitted in full, accompanied by the carrier wave. 

Accordingly, demodulation of an AM signal is accomplished rather simply in the receiver 
by using an envelope detector, for example. It is for this reason we find that full AM is 
commonly used in commercial AM radio broadcasting, which involves a single powerful 
transmitter and numerous receivers that are relatively inexpensive to build. 
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2. Double sideband-suppressed carrier (DSB-SC) modulation, in which only the upper 
and lower sidebands are transmitted. 

The suppression of the carrier wave means that DSB-SC modulation requires much less 
power than full AM to transmit the same message signal; this advantage of DSB-SC mod- 
ulation over full AM is, however, attained at the expense of increased receiver complexity. 
DSB-SC modulation is therefore well suited for point-to-point communication involving 
one transmitter and one receiver; in this form of communication, transmitted power is at 
a premium and the use of a complex receiver is therefore justifiable. 

3. Single sideband (SSB) modulation, in which only the upper sideband or lower sideband 
is transmitted, 

SSB modulation is the optimum form of CW modulation in the sense that it requires the 
minimum transmitted power and the minimum channel bandwidth for conveying a mes- 
sage signal from one point to another. However, its use is limited to message signals with 
an energy gap centered on zero frequency. 

4. Vestigial sideband modulation, in which almost all of one sideband and a vestige of 
the other sideband are transmitted in a prescribed complementary fashion. 

VSB modulation requires a channel bandwidth that is between that required for SSB and 
DSB-SC systems, and the saving in bandwidth can be significant if modulating signals with 
large bandwidths are being handled, as in the case of television signals and high-speed 
data. 

DSB-SC, SSB, and VSB are examples of linear modulation, whereas, strictly speaking, full 
AM is nonlinear. However, the deviation of full AM from linearity is of a mild sort. 
Accordingly, all four forms of amplitude modulation lend themselves readily to spectral 
analysis using the Fourier transform. 

■ Angle Modulation 

Angle modulation may be classified into frequency modulation (FM) and phase modula- 
tion (PM). In FM, the instantaneous frequency of a sinusoidal carrier is varied in propor- 
tion to the message signal. In FM, on the other hand, it is the phase of the carrier that is 
varied in proportion to the message signal. The instantaneous frequency is defined as the 
derivative of the phase with respect to time, except for the scaling factor 1/{2it). Accord- 
ingly, FM and PM are closely related to each other; if we know the properties of the one, 
we can determine those of the other. For this reason, and because FM is commonly used 
in broadcasting, much of the material on angle modulation in the chapter was devoted 
to FM. 

Unlike amplitude modulation, FM is a nonlinear modulation process. Accordingly, 
spectral analysis of FM is more difficult than for AM. Nevertheless, by studying single- 
tone FM, we were able to develop a great deal of insight into the spectral properties of 
FM. In particular, we derived an empirical rule known as Carson's rule for an approximate 
evaluation of the transmission bandwidth B T of FM. According to this rule, B T is controlled 
by a single parameter: the modulation index |3 for sinusoidal FM, or the deviation ratio 
D for nonsinusoidal FM. 

■ Noise Analysis 

We conclude the chapter on CW modulation systems by presenting a comparison of their 
noise performances. For this comparison, we assume that the modulation is produced by 
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Channel signal-to-noise ratio, dB 

Figure 2.55 Comparison of the noise performance of various CW modulation systems. Curve [; 
Full AM, ix = 1. Curve II: DSB-SC, SSB. Curve III: FM, /3 = 2. Curve IV: FM, /3 = 5. (Curves 
III and IV include 13-dB pre-emphasis, de-emphasis improvement.) 



a sinusoidal wave. For the comparison to be meaningful, we also assume that the modu- 
lation systems operate with exactly the same channel signal-to-noise ratio. We may thus 
plot the output signal-to-noise ratio versus the channel signal-to-noise ratio as in Figure 
2.55 for the following modulation schemes: 
*■ Full AM with 100 percent modulation 

► Coherent DSB-SC, SSB 

► FM with /3 = 2 and /3 = 5 

Figure 2.55 also includes the AM and FM threshold effects. In making the comparison, it 
is informative to keep in mind the transmission bandwidth requirement of the modulation 
system in question. In this regard, we use a normalized transmission bandwidth denned by 

where B T is the transmission bandwidth of the modulated signal, and W is the message 
bandwidth. Table 2.4 presents the values of B„ for the different CW modulation schemes. 

From Figure 2.55 and Table 2.4 we make the following observations: 
t. Among the family of AM systems, SSB modulation is optimum with regard to noise 

performance as well as bandwidth conservation. 

► The use of FM improves noise performance but at the expense of an excessive trans- 
mission bandwidth. This assumes that the FM system operates above threshold for 
the noise improvement to be realized. 
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TABLE 2.4 Values of B„for 
various CW modulation schemes 







FM 




AM, DSB-SC SSB 


3=2 p=5 




2 1 


8 16 



On an important point to conclude the discussion on CW modulation, only frequency 
modulation offers the capability to trade off transmission bandwidth for improved noise 
performance. The trade-off follows a square law, which is the best that we can do with 
CW modulation (i.e., analog communications). In Chapter 3 we describe pulse-code mod- 
ulation, which is basic to the transmission of analog information-bearing signals by a 
digital communication system, and which can indeed do much better. 
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Amplitude Modulation 

2.1 Suppose that nonlinear devices are available for which the output current i 0 and input 
voltage Vj are related by 



where a j and a 3 are constants. Explain how these devices may be used to provide: (a) a 
product modulator and (b) an amplitude modulator. 
2.2 Figure P2.2 shows the circuit diagram of a square-law modulator. The signal applied to 
the nonlinear device is relatively weak, such that it can be represented by a square law: 



where a t and a z are constants, Vi(t) is the input voltage, and is the output voltage. 
The input voltage is denned by 



where m(t) is a message signal and A c cos(2irf c £) is the carrier wave. 

(a) Evaluate the output voltage v^t). 

(b) Specify the frequency response that the tuned circuit in Figure P2.2 must satisfy in 
order to generate an AM signal with f c as the carrier frequency. 

(c) What is the amplitude sensitivity of this AM signal? 



Problems 



i„ = a t v, + a 3 vf 



v 2 {t) = a lVl (t) + a 2 vl{t) 



Vi(t) = A c cos(2ir/ e f) + m(t) 




Tuned to f c 



Figure P2.2 



2.3 Figure P2.3a shows the circuit diagram of a switching modulator. Assume that the earner 
wave c(t) applied to the diode is large in amplitude, so that the diode acts like an ideal 
switch: it presents zero impedance when forward biased (i.e., c(t) > 0). We may thus 
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approximate the transfer characteristic of the diode-load resistor combination by a piece- 
wise-linear characteristic defined as (see Figure V23b) 

( Vl (t), c(t) > 0 

^ ) = lo, *)<0 

That is, the load voltage v 2 (t) varies periodically between the values Vi(t) and zero at a 
rate equal to the carrier frequency f c . Hence, we may write 

Mt) = [A e cos(2nf c t) + m(t)]g To (t} 

where gT a (t) is a periodic pulse train defined by 

gr 0 (t) = i + - E cos[2i7fr<2» - 1)] 

1 it ra- i Ln — 1 

(a) Find the AM wave component contained in the output voltage v 2 {t). 

(b) Specify the unwanted components in v 2 (t) that need to be removed by a band-pass 
filter of suitable design. 



c(() = A c COS (2ir£<) 

■cvj — t — N- 




v x {t) Rj > v z (t) 




Figure P2.3 



2.4 Consider the AM signal 

s(t) = A e [l + p- cos(2ir/„,f)] cos(2ir/ e () 

produced by a sinusoidal modulating signal of frequency /„. Assume that the modulation 
factor is p = 2, and the carrier frequency f c is much greater than /„. The AM signal s(t) 
is applied to an ideal envelope detector, producing the output v(t). 

(a) Determine the Fourier series representation of v(t). 

(b) What is the ratio of second-harmonic amplitude to fundamental amplitude in v(t)1 

2.5 Figure P2.5 shows the circuit diagram of an envelope detector. It consists simply of a 
diode and resistor-capacitor (RC) filter. On a positive half-cycle of the input signal, the 
diode is forward-biased and the capacitor C charges up rapidly to the peak value of the 
input signal. When the input signal falls below this value, the diode becomes reverse- 
biased and the capacitor C discharges slowly through the load resistor R ; . The discharging 
process continues until the next positive half-cycle. Thereafter, the charging-discharging 
routine is continued. 

(a) Specify the condition that must be satisfied by the capacitor C for it to charge rapidly 
and thereby follow the input voltage up to the positive peak when the diode is 
conducting. 

(b) Specify the condition which the load resistor R, must satisfy so that the capacitor C 
discharges slowly between positive peaks of the carrier wave, but not so long that the 
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capacitor voltage will not discharge at the maximum rate of change of the modulating 



AM wave / 
id) 



C =fc R; > Output 



Figure P2.5 



2.6 Consider a square-law detector, using a nonlinear device whose transfer characteristic is 
denned by 

v 2 (f) = a lVl {t) + a z v\(t) 

where a t and a 2 are constants, Vl (t) is the input, and v 2 (t) is the output. The input consists 
of the AM wave 

Vl (t) = AJ1 + fe„m(f)] cos(2ir/ c f) 

(a) Evaluate the output v z (t). 

(b) Find the conditions for which the message signal m(t) may be recovered from u 2 (t). 

2.7 The AM signal 

s(t) = A e [l + W)] cos(2ir/ e r) 

is applied to the system shown in Figure P2.7. Assuming that | k a m(t) I < 1 for all t and 
the message signal m(t) is limited to the interval -W < / s W and that the earner 
frequency /„ > 2W show that m(t) can be obtained from the square-rooter output u 3 (f|. 



s(t) - 





u,(r) 




v 2 {t) 




Squarer 


Low-pass 


Square 




filter 




rooter 



u 3 w = Vm^ 



Figure P2.7 



2.8 Consider a message signal m(t) with the spectrum shown m Figure P2.8. The message 
bandwidth W = 1 kHz. This signal is applied to a product modulator, together witn a 
carrier wave A c cos(2ir/ e (), producing the DSB-SC modulated signal s(t). The modula ed 
signal is next applied to a coherent detector. Assuming perfect synchronism between m 
carrier waves in the modulator and detector, determine the spectrum of the detecro 
output when: (a) the carrier frequency f c = 1.25 kHz and (b) the carrier frequency 
f c = 0.75 kHz. What is the lowest carrier frequency for which each component oi u 
modulated signal s(t) is uniquely determined by tn(t)l 
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-WOW 
Figure P2.8 



2.9 Figure P2.9 shows the circuit diagram of a balanced modulator. The input applied to the 
top AM modulator is m(t), whereas that applied to the lower AM modulator is —m{t); 
these two modulators have the same amplitude sensitivity. Show that the output s(t) of 
the balanced modulator consists of a DSB-SC modulated signal. 



AM 
modulator 



A c cos (2irf c t) 



A c cos (2ir/ c r) 



AM -'2 M 
modulator 



Figure P2.9 



2.10 A DSB-SC modulated signal is demodulated by applying it to a coherent detector. 

(a) Evaluate the effect of a frequency error A/ in the local carrier frequency of the de- 
tector, measured with respect to the carrier frequency of the incoming DSB-SC signal. 

(b) For the case of a sinusoidal modulating wave, show that because of this frequency 
error, the demodulated signal exhibits beats at the error frequency. Illustrate your 
answer with a sketch of this demodulated signal. 

2.11 Consider the DSB-SC signal 

s(f) = A c cos(lTTf c t)m(t) 

where A c cos(27r/ 6 t) is the carrier wave and m(t) is the message signal. This modulated 
signal is applied to a square-law device characterized by 

y(t) = s 2 (t) 

The output y(t) is next applied to a narrowband filter with a passband magnitude response 
of one, midband frequency 2/ c , and bandwidth A/. Assume that A/ is small enough to 
treat the spectrum of y(t) as essentially constant inside the passband of the filter. 

(a) Determine the spectrum of the square-law device output y(t). 

(b) Show that the filter output v(t) is approximately sinusoidal, given by 



v(t) 



< y E A/ cos(47r/ c f) 



where E is the energy of the message signal m(t). 
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2.12 Consider the quadrature-carrier multiplex system of Figure 2.10. The multiplexed signal 
s(t) produced at the transmitter output in Figure 2.10a is applied to a communication 
channel of frequency response H(f). The output of this channel is, in turn, applied to the 
receiver input in Figure 2.10b. Prove that the condition 

H(f c + f) = H"(f c - /), 0 < / £ W 

is necessary for recovery of the message signals m^t) and m 2 {t) at the receiver outputs; 
f c is the carrier frequency, and W is the message bandwidth. Hint: Evaluate the spectra 
of the two receiver outputs. 

2.13 Suppose that in the receiver of the quadrature-carrier multiplex system of Figure 2.1 Of, 
the local carrier available for demodulation has a phase error <j> with respect to the carrier 
source used in the transmitter. Assuming a distortionless communication channel between 
transmitter and receiver, show that this phase error will cause cross-talk to arise between 
the two demodulated signals at the receiver outputs. By cross-talk we mean that a portion 
of one message signal appears at the receiver output belonging to the other message signal, 
and vice versa. 

2.14 A particular version of AM stereo uses quadrature multiplexing. Specifically, the carrier 
A c cos(2ir/ 6 t) is used to modulate the sum signal 

m a (r) = V 0 + m,{t) + m r {t) 

where V 0 is a DC offset included for the purpose of transmitting the carrier component, 
m t (t) is the left-hand audio signal, and m r (t) is the right-hand audio signal. The quadrature 
carrier A c s\a(2irf c t) is used to modulate the difference signal 

mjyi) = m,{t) - m T (i) 

(a) Show that an envelope detector may he used to recover the sum m r {t) + m t (t) from 
the quadrature-multiplexed signal. How would you minimize the signal distortion 
produced by the envelope detector? 

(b) Show that a coherent detector can recover the difference m e (t) - m r (t). 

(c) How are the desired m t (t) and m r (t) finally obtained? 

2.15 Using the message signal 

determine and sketch the modulated waves for the following methods of modulation: 

(a) Amplitude modulation with 50 percent modulation. 

(b) Double sideband-suppressed carrier modulation. 

(c) Single sideband modulation with only the upper sideband transmitted. 

(d) Single sideband modulation with only the lower sideband transmitted. 

2.16 The Hilbert transform of a Fourier transformable signal m(t), denoted by m(t), is defined 
by (see Appendix 2) 

1 P ™{t) , 
rh{t) = - — — dr 

W 77 J — so t T 

In the frequency domain, we have 

M(f) = -j sgn(/)M(/) 



where m(t) ^ M(f), m(t) ^ M{f), and sgn(/) is the signum function. 



Problems 171 



Using the definition of the Hilbert transform, show that a single-sideband modu- 
lated signal resulting from the message signal tn(t) and carrier cos(2irf c t) of unit amplitude 
is given by (see Table 2.1} 

s(t) = | m(t) cos(2Trf c t) ± i m(t) sin(2ir£f) 

where the minus sign corresponds to the transmission of the upper sideband and the plus 
sign corresponds to the transmission of the lower sideband. 

2.17 The local oscillator used for the demodulation of an SSB signal s(t) has a frequency error 
A/ measured with respect to the carrier frequency f c used to generate s{t). Otherwise, 
there is perfect synchronism between this oscillator in the receiver and the oscillator sup- 
plying the carrier wave in the transmitter. Evaluate the demodulated signal for the fol- 
lowing two situations: 

(a) The SSB signal s(t) consists of the upper sideband only. 

(b) The SSB signal s(t) consists of the lower sideband only. 

2.18 Figure P2.1 S shows the block diagram of Weaver's method for generating SSB modulated 
waves. The message (modulating) signal m(t) is limited to the band f a < | f\ £ /;,. The 
auxiliary carrier applied to the first pair of product modulators has a frequency f 0 , which 
lies at the center of this band, as shown by 




The low-pass filters in the in-phase and quadrature channels are identical, each with a 
cutoff frequency equal to (/;, — /J/2. The carrier applied to the second pair of product 
modulators has a frequency f c that is greater than (/;, — f„)l2. Sketch the spectra at the 
various points in the modulator of Figure P2.18, and hence show that: 

(a) For the lower sideband, the contributions of the in-phase and quadrature channels 
are of opposite polarity, and by adding them at the modulator output, the lower 
sideband is suppressed. 

(b) For the upper sideband, the contributions of the in-phase and quadrature channels 
are of the same polarity, and by adding them, the upper sideband is transmitted. 

(c) How would you modify the modulator of Figure P2.18 so that only the lower side- 
band is transmitted? 
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Quadrature channel 
Figure P2.18 
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2.19 The spectrum of a voice signal m{t) is zero outside the interval /„ =s | / 1 := f b . To ensure 
communication privacy, this signal is applied to a scrambler that consists of the following 
cascade of components: product modulator, high-pass filter, second product modulator, 
and low-pass filter. The carrier wave applied to the first product modulator has a fr e . 
quency equal to f c , whereas that applied to the second product modulator has a frequency 
equal to + f c ; both of them have unit amplitude. The high-pass and low-pass filters 
have the same cutoff frequency at f. Assume that f > ft,. 

(a) Derive an expression for the scrambler output s(f), and sketch its spectrum. 

(b) Show that the original voice signal m{t) may be recovered from s{t) by using an 
unscrambler that is identical to the unit just described. 

2.20 In this problem we derive Equation (2.16) that defines the frequency response H Q (f) 0 f 
the filter that operates on the message signal m{t) to produce m'(t) for VSB modulation. 
The signal m'(t), except for a scaling factor, constitutes the quadrature component of s(t). 
To do the derivation, we apply s(f) to the coherent detector of Figure P2.20 so as to 
recover a scaled version of the original message signal m(t). 

(a) Starting with the block diagram of Figure 2.12 for the generation of a VSB modulated 
wave, determine the Fourier transform V(f) of the product modulator output v(t) ij 
Figure P2.20 in terms of the Fourier transform of the message signal m{t) and the 
frequency response H{f) of the band-pass filter in Figure 2.12. 

(b) Hence, by evaluating the Fourier transform of the low-pass filter output in Figure 
P2.20, determine the conditions that must be satisfied by H Q (f) in terms of H(f) to 
assure perfect recovery of the original message signal m(t), except for a scaling factor. 
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filter 
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A c cos (2u/ c rt 

Figure P2.20 



2.21 The single-tone modulating signal m(t ) = A m cos(2ir/„,f) is used to generate the VSB signal 

s(() = i MA cos[2-n-(/ e + fjt] + i AJi c (l - a) cos[2ir(/ c - fjt] 

where a is a constant, less than unity, representing the attenuation of the upper side 
frequency. 

(a) Find the quadrature component of the VSB signal s(f). 

(b) The VSB signal, plus the carrier A c cos(2irf c t), is passed through an envelope detector. 
Determine the distortion produced by the quadrature component. 

(c) What is the value of constant a for which this distortion reaches its worst possible 
condition? 

2.22 In this problem we study the idea of mixing in a superheterodyne receiver. To be specific, 
consider the block diagram of the mixer shown in Figure P2.22 that consists of a product 
modulator with a local oscillator of variable frequency f h followed by a band-pass filter- 
The input signal is an AM wave of bandwidth 10 kHz and carrier frequency that may lie 
anywhere in the range of 0.535 to 1.605 MHz; these parameters are typical of AM radio 
broadcasting. It is required to translate this signal to a frequency band centered at a fixed 
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intermediate frequency (IF) of 0.455 MHz. Find the range of tuning that must be provided 
in the local oscillator to achieve this requirement. 




cos (2ir/ f () - 

Figure P2.22 



2.23 Figure P2.23 shows the block diagram of a heterodyne spectrum analyzer. It consists of 
a variable-frequency oscillator, multiplier, band-pass filter, and root mean square (RMS) 
meter. The oscillator has an amplitude A and operates over the range f 0 to / 0 + W, where 
/o is the midband frequency of the filter and W is the signal bandwidth. Assume that 
/o = 2W, the filter bandwidth A/ is small compared with f 0 and that the passband mag- 
nitude response of the filter is one. Determine the value of the RMS meter output for a 
low-pass input signal g(t). 



Input 
signal gU) 




Band-pass 




RMS 


filter 




meter 



Output 
* signal 



Figure P2.23 

Angle Modulation 

2.24 Sketch the PM and FM waves produced by the sawtooth wave shown in Figure P2.24. 




To 2T 0 
Figure P2.24 



2.25 In a frequency-modulated radar, the instantaneous frequency of the transmitted carrier 
is varied as in Figure P2.25, which is obtained by using a triangular modulating signal. 
The instantaneous frequency of the received echo signal is shown dashed in Figure P2.25, 
where ris the round-trip delay time. The transmitted and received echo signals are applied 
to a mixer, and the difference frequency component is retained. Assuming that f 0 T « 1, 
determine the number of beat cycles at the mixer output, averaged over one second, in 
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terms of the peak deviation A/ of the carrier frequency, the delay t, and the repetition 
frequency f a of the transmitted signal. 




Figure P2.25 



2.26 The instantaneous frequency of a sine wave is equal to f c - A/ for 1 1 | s T/2, and f c for 
1 1 1 > T/2. Determine the spectrum of this frequency-modulated wave. Hint: Divide up 
the time interval of interest into three regions: — <*> < t < -T/2, -T/2 S(£ T/2, and 
T/2<t< °°. 

2.27 Single-sideband modulation may be viewed as a hybrid form of amplitude modulation 
and frequency modulation. Evaluate the envelope and instantaneous frequency of anSSB 
wave for the following two cases: 

(a) When only the upper sideband is transmitted. 

(b) When only the lower sideband is transmitted. 

2.28 Consider a narrowband FM signal approximately denned by 

s{t) = A c cos(2-irfj) - pA e sin(2ir£f) sin(27r/ m r) 

(a) Determine the envelope of this modulated signal. What is the ratio of the maximum 
to the minimum value of this envelope? Plot this ratio versus ft assuming that /3 is 
restricted to the interval 0s^< 0.3. 

(b) Determine the average power of the narrowband FM signal, expressed as a percentage 
of the average power of the unmodulated carrier wave. Plot this result versus ft 
assuming that j6 is restricted to the interval 0<^s 0.3. 

(c) By expanding the angle 0,-(t) of the narrow-band FM signal s(«) in the form of a power 
series, and restricting the modulation index /3 to a maximum value of 0.3 radians, 
show that 

fl 3 

0,(f) = 2-rrfJ + $ sin(2ir/ m t) - y sin 3 (2ir/„r-) 

What is the power ratio of third harmonic to fundamental component for (3 = 0.3? 

2.29 The sinusoidal modulating wave 

m{t) = A„, cos(27r/„f) 

is applied to a phase modulator with phase sensitivity k p . The unmodulated carrier wave 
has frequency f c and amplitude A c . 

(a) Determine the spectrum of the resulting phase-modulated signal, assuming that the 
maximum phase deviation /3 P — kpA m does not exceed 0.3 radians. 

(b) Construct a phasor diagram for this modulated signal, and compare it with that of 
the corresponding narrowband FM signal. 

2.30 Suppose that the phase-modulated signal of Problem 2.29 has an arbitrary value for the 
maximum phase deviation jS p . This modulated signal is applied to an ideal band-pass filter 
with midband frequency f c and a passband extending from f c - 1.5 f m to f c + 
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Determine the envelope, phase, and instantaneous frequency of the modulated signal at 
the filter output as functions of time. 

2.31 A carrier wave is frequency-modulated using a sinusoidal signal of frequency f„ and 
amplitude A m . 

(a) Determine the values of the modulation index for which the carrier component of 
the FM signal is reduced to zero. For this calculation you may use the values of / 0 (/3) 
given in Table A6.5. 

(b) In a certain experiment conducted with /„ = 1 kHz and increasing A m (starting from 
0 volts), it is found that the carrier component of the FM signal is reduced to zero 
for the first time when A„ = 2 volts. What is the frequency sensitivity of the modu- 
lator? What is the value of A m for which the carrier components is reduced to zero 
for the second time? 

2.32 An FM signal with modulation index /3 = 1 is transmitted through an ideal band-pass 
filter with midband frequency f c and bandwidth 5/ m , where f c is the carrier frequency and 
f m is the frequency of the sinusoidal modulating wave. Determine the magnitude spectrum 
of the filter output. 

2.33 A carrier wave of frequency 100 MHz is frequency-modulated by a sinusoidal wave of 
amplitude 20 volts and frequency 100 kHz. The frequency sensitivity of the modulator 
is 25 kHz per volt. 

(a) Determine the approximate bandwidth of the FM signal, using Carson's rule. 

(b) Determine the bandwidth by transmitting only those side frequencies whose ampli- 
tudes exceed 1 percent of the unmodulated carrier amplitude. Use the universal curve 
of Figure 2.26 for this calculation. 

(c) Repeat your calculations, assuming that the amplitude of the modulating signal is 
. doubled. 

(d) Repeat your calculations, assuming that the modulation frequency is doubled. 

2.34 Consider a wideband PM signal produced by a sinusoidal modulating wave 
A m cos(2ir/„f), using a modulator with a phase sensitivity equal to k p radians per volt. 

(a) Show that if the maximum phase deviation of the PM signal is large compared with 
one radian, the bandwidth of the PM signal varies linearly with the modulation fre- 
quency /„. 

(b) Compare this characteristic of a wideband PM signal with that of a wideband FM 
signal. 

2.35 Figure P2.35 shows the block diagram of a real-time spectrum analyzer working on the 
principle of frequency modulation. The given signal g(t) and a frequency-modulated signal 
s(f) are applied to a multiplier and the output g(t)s[t) is fed into a filter of impulse response 
h{t). The s(t ) and h{t) are linear FM signals whose instantaneous frequencies vary linearly 
with time at opposite rates, as shown by 

s(f) = cos(2ir/ c f - irkt 1 ) 
h(t) = cos(2-n/ e f + irkt 2 ) 

where k is a constant. Show that the envelope of the filter output is proportional to the 
magnitude spectrum of the input signal g(t) with kt playing the role of frequency /. Hint: 
Use the complex notations described in Appendix 2 for the analysis of band-pass signals 
and band-pass filters. 




Output 
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Figure P2.35 
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2.36 An FM signal with a frequency deviation of 10 kHz at a modulation frequency of 5 kHz 
is applied to two frequency multipliers connected in cascade. The first multiplier doubles 
the frequency and the second multiplier triples the frequency. Determine the frequency 
deviation and the modulation index of the FM signal obtained at the second multiplier 
output. What is the frequency separation of the adjacent side frequencies of this FM 
signal? 

2.37 Figure P2.37 shows the block diagram of a wideband frequency modulator using the 
indirect method. This modulator is used to transmit audio signals containing frequencies 
in the range of 100 Hz to 15 kHz. The narrowband phase modulator is supplied with a 
carrier of frequency / a = 0.1 MHz by a crystal-controlled oscillator. A second crystal- 
controlled oscillator supplies a sinusoidal wave of frequency 9.5 MHz to the mixer. The 
system specifications are as follows: 

Carrier frequency at the transmitter output, f c = 100 MHz 

Minimum frequency deviation, A/ = 75 kHz 

Maximum modulation index in the phase modulator = 0.3 radians 

(a) Calculate the frequency multiplication ratios n a and n 2 (preceding and following the 
mixer), which will satisfy these specifications. 

(b) Specify the values of the carrier frequency and frequency deviation at the various 
points in the modulator of Figure P2.37. 
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2.38 Figure P2.3 8 shows the frequency-determining network of a voltage-controlled oscillator. 
Frequency modulation is produced by applying the modulating signal A„ sin(2u/„f) plus 
a bias V b to a pair of varactor diodes connected across the parallel combination of a 
200-jiH inductor and 100-pF capacitor. The capacitor of each varactor diode is related 
to the voltage V (in volts) applied across its electrodes by 

C = lOOV^ 1 ' 2 pF 

The unmodulated frequency of oscillation is 1 MHz. The VCO output is applied to a 
frequency multiplier to produce an FM signal with a carrier frequency of 64 MHz and a 
modulation index of 5. Determine (a) the magnitude of the bias voltage V b and (b) the 
amplitude A,„ of the modulating wave, given that /„ = 10 kHz. 
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2.39 The FM signal 



s{t) = A c cos 



lTrf c t + 2irkf m(r) dr 



is applied to the system shown in Figure P2.39 consisting of a high-pass RC filter and an 
envelope detector. Assume that (a) the resistance R is small compared with the reactance 
of the capacitor C for all significant frequency components of s(t), and (b) the envelope 
detector does not load the filter. Determine the resulting signal at the envelope detector 
output, assuming that k f \m{t) \ < f c for all t. 
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2.40 Consider the frequency demodulation scheme shown in Figure P2.40 in which the incom- 
ing FM signal s(t) is passed through a delay line that produces a phase-shift of 7r/2 radians 
at the carrier frequency f c . The delay-line output is subtracted from the incoming FM 
signal, and the resulting composite signal is then envelope-detected. This demodulator 
finds application in demodulating microwave FM signals. Assuming that 

s{t) = A c cos[2ir/ c f + /} sin(2-n-/„t)] 

analyze the operation of this demodulator when the modulation index /8 is less than unity 
and the delay T produced by the delay line is sufficiently small to justify making the 
approximations 

cos(2ir/„T) = 1 

and 

sin(2i7/ m T) = 2irf„T 




2.41 Figure 2.41 shows the block diagram of a zero-crossing detector for demodulating an FM 
signal. It consists of a limiter, a pulse generator for producing a short pulse at each zero- 
crossing of the input, and a low-pass filter for extracting the modulating wave, 
(a) Show that the instantaneous frequency of the input FM signal is proportional to the 

number of zero crossings in the time interval t — (Tj/2) to t + {T^/2), divided by T t . 

Assume that the modulating signal is essentially constant during this time interval. 
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(b) Illustrate the operation of this demodulator, using the sawtooth wave of Figure V2 j.^ 
as the modulating wave. 
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2.42 Suppose that the received signal in an FM system contains some residual amplitude mod- 
ulation of positive amplitude a(t), as shown by 

s(t) = a(t) cos[2ir/ e f + 0(f)] 

where f c is the carrier frequency. The phase 0(f) is related to the modulating signal m(f) 
by 

0(f) = 2irfe f m{r) dr 

Jo 

where k f is a constant. Assume that the signal s(f) is restricted to a frequency band of 
width B T , centered at /„, where B T is the transmission bandwidth of the FM signal in the 
absence of amplitude modulation, and that the amplitude modulation is slowly varying 
compared with 0(f). Show that the output of an ideal frequency discriminator produced 
by s(f) is proportional to a(t)m(t). Hint: Use the complex notation described in Appendix 
2 to represent the modulated wave s(t). 

2.43 (a) Let the modulated wave s(f) in Problem 2.42 be applied to a hard limiter, whose 

output z{t) is defined by 

z(t) = sgn[s(f)] 

_ f + 1, s(t) > 0 
~~ 1-1, s(t) < 0 

Show that the limiter output may be expressed in the form of a Fourier series as 
follows: 

4t) = - 2) T-r^r cos[2 T / e f(2n + 1) + (2» + l)0(f)] 

(b) Suppose that the limiter output is applied to a band-pass filter with a passband mag- 
nitude response of one and bandwidth B T centered about the carrier frequency /„ 
where B T is the transmission bandwidth of the FM signal in the absence of amplitude 
modulation. Assuming that f c is much greater than B T , show that the resulting filter 
output equals 

y(t) = - cos[2 T / e f + 0(f)] 

TT 

By comparing this output with the original modulated signal s(t) defined in Problem 
2.42, comment on the practical usefulness of the result. 

2.44 (a) Consider an FM signal of carrier frequency /„ which is produced by a modulating 

signal m(t). Assume that f c is large enough to justify treating this FM signal as a 
narrowband signal. Find an approximate expression for its Hilbert transform, 
(b) For the special case of a sinusoidal modulating wave m(t) = A m cos(2 ir/„,f), find the 
exact expression for the Hilbert transform of the resulting FM signal. For this case, 
what is the error in the approximation used in part (a)? 
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2.45 The single sideband version of angle modulation is defined by 

s(t) = exp[-<£(t)] cos[2i7/ c f + 4>(t)] 

where <t>(t) is the Hilbert transform of the phase function <t>(t), and / c is the carrier 
frequency. 

(a) Show that the spectruin of the modulated signal s(t) contains no frequency compo- 
nents in the interval —f c < f < f„ and is of infinite extent. 

(b) Given that the phase function 

4>(t) = P sin(2u/„,r) 

where /3 is the modulation index and f m is the modulation frequency, derive the 
corresponding expression for the modulated wave s(t). 

Note: For Problems 2.44 and 2.45, you need to refer to Appendix 2 for a treatment of 
the Hilbert transform. 

Noise in CW Modulation Systems 

2.46 A DSB-SC modulated signal is transmitted over a noisy channel, with the power spectral 
density of the noise being as shown in Figure P2.46. The message bandwidth is 4 kHz 
and the carrier frequency is 200 kHz. Assuming that the average power of the modulated 
wave is 10 watts, determine the output signal-to-noise ratio of the receiver. 




2.47 Evaluate the autocorrelation functions and cross-correlation functions of the in-phase and 
quadrature components of the narrowband noise at the coherent detector input for 
(a) the DSB-SC system, (b) an SSB system using the lower sideband, and (c) an SSB system 
using the upper sideband. 

2.48 In a receiver using coherent detection, the sinusoidal wave generated by the local oscillator 
suffers from a phase error 8(t) with respect to the carrier wave cos{2i7/ e f). Assuming that 
8(t) is a sample function of a zero-mean Gaussian process of variance a%, and that most 
of the time the maximum value of 0(f) is small compared with unity, find the mean-square 
error of the receiver output for DSB-SC modulation. The mean-square error is defined as 
the expected value of the squared difference between the receiver output and the message 
signal component of the receiver output. 

2.49 Following a procedure similar to that described in Section 2.11 for the DSB-SC receiver, 
extend this noise analysis to a SSB receiver. Specifically, evaluate the following; 

(a) The output signal-to-noise ratio. 

(b) The channel signal-to-noise ratio. 

Hence, show that the figure of merit for the SSB receiver is exactly the same as that for 
the DSB-SC receiver. Note that unlike the DSB-SC receiver, the midband frequency of the 
spectral density function of the narrowband-filtered noise at the front end of the SSB 
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receiver is offset from the carrier frequency f c by an amount equal to W/2, where W j s 
the message bandwidth. 

2.50 Let a message signal m{t) be transmitted using single-sideband modulation. The power 
spectral density of m(t) is 

fill I/I=SW 

s M (f) = \ W ' 

(_0, otherwise 

where a and W are constants. White Gaussian noise of zero mean and power spectral 
density N 0 /2 is added to the SSB modulated wave at the receiver input. Find an expression 
for the output signal-to-noise ratio of the receiver. 

2.51 Consider the output of an envelope detector defined by Equation (2.92), which is repro- 
duced here for convenience 

y(t) = {[A c + AJk a m(t) + ni (t)f + n 2 Q (t)} in 

(a) Assume that the probability of the event 

\n Q (t)\ > eA c \l + kjn(t)\ 

is equal to or less than S,, where s « 1. "What is the probability that the effect of 
the quadrature component »q(J) is negligible? 

(b) Suppose that k a is adjusted relative to the message signal m(t) such that the probability 
of the event 

A c [l + k a m{t)] + n,(t) < 0 
is equal to 82. What is the probability that the approximation 
y{t) = A c [l + k a m(t)] + n,(t) 

is valid? 

(c) Comment on the significance of the result in part (b) for the case when &, and are 
both small compared with unity. 

2.52 An unmodulated carrier of amplitude A c and frequency f c and band-limited white noise 
are summed and then passed through an ideal envelope detector. Assume the noise spec- 
tral density to be of height N 0 /2 and bandwidth 2 W, centered about the carrier frequency 
f c . Determine the output signal-to-noise ratio for the case when the carrier-to-noise ratio 
is high. 

2.53 Let R denote the random variable obtained by observing the output of an envelope de- 
tector at some fixed time. Intuitively, the envelope detector is expected to be operating 
well into the threshold region if the probability that the random variable R exceeds the 
carrier amplitude A c is 0.5. On the other hand, if this same probability is only 0.01, the 
envelope detector is expected to be relatively free of loss of message and the threshold 
effect. 

(a) Assuming that the narrowband noise at the detector input is white, zero-mean, Gaus- 
sian with spectral density N 0 /2 and the message bandwidth is W, show that the prob- 
ability of the event R a A c is 

P(R a A„) = exp(- p) 
where p is the carrier-to-noise ratio: 



r 4WN 0 

(b) Using the formula for this probability, calculate the carrier-to-noise ratio when (1) the 
envelope detector is expected to be well into the threshold region, and (2) it is expected 
to be operating satisfactorily. 
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2.54 Consider a phase modulation (PM) system, with the modulated wave denned by 

s{t) = A c cos[2irf c t + k p m(t)} 

where k p is a constant and m(t) is the message signal. The additive noise n(t) at the phase 
detector input is 

n(t) = n t (t) cos(2irf c t) - n Q (t) sin(2irf c t) 

Assuming that the carrier-to-noise ratio at the detector input is high compared with unity, 
determine (a) the output signal-to-noise ratio and (b) the figure of merit of the system. 
Compare your results with the FM system for the case of sinusoidal modulation. 

2.55 An FDM system uses single-sideband modulation to combine 12 independent voice signals 
and then uses frequency modulation to transmit the composite baseband signal. Each 
voice signal has an average power P and occupies the frequency band 0.3 to 3,4 kHz; the 
system allocates it a bandwidth of 4 kHz. For each voice signal, only the lower sideband 
is transmitted. The subcarrier waves used for the first stage of modulation are defined by 

c k {t) = A k cos(2irkf 0 t), 0 < k < 11 

The received signal consists of the transmitted FM signal plus white Gaussian noise of 
zero mean and power spectral density N 0 /2. 

(a) Sketch the power spectral density of the signal produced at the frequency discrimi- 
nator output, showing both the signal and noise components. 

(b) Find the relationship between the subcarrier amplitudes A k so that the modulated 
voice signals have equal signal-to-noise ratios. 

2. 56 In the discussion on FM threshold effect presented in Section 2.13, we described the 
conditions for positive-going and negative-going clicks in terms of the envelope r(t) and 
phase ij/(t) of the narrowband noise n(t). Reformulate these conditions in terms of the in- 
phase component « ; (t) and quadrature component ng(t) of n(t). 

2.57 By using the pre-emphasis filter shown in Figure 2.50a and with a voice signal as the 
modulating wave, an FM transmitter produces a signal that is essentially frequency- 
modulated by the lower audio frequencies and phase-modulated by the higher audio 
frequencies. Explain the reasons for this phenomenon. 

2.58 Suppose that the transfer functions of the pre-emphasis and de-emphasis filters of an FM 
system are scaled as follows: 

H^/) - *(l + |) 

and 

The scaling factor k is to be chosen so that the average power of the emphasized message 
signal is the same as that of the original message signal m(t). 

(a) Find the value of k that satisfies this requirement for the case when the power spectral 
density of the message signal m(t) is 

[0, elsewhere 

(b) What is the corresponding value of the improvement factor I produced by using this 
pair of pre-emphasis and de-emphasis filters? Compare this ratio with that obtained 
in Example 2.6. The improvement factor I is defined by Equation (2.160). 
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2.59 A phase modulation (PM) system uses a pair of pre-emphasis and de-emphasis filters 
defined by the transfer functions 

Hp.CZ) = 1 + 7 
Jo 

and 

1 



H de (/) l + {jm 

Show that the improvement in output signal-to-noise ratio produced by using this pair of 
filters is 

W/f 0 



tan-'(Wo) 

where W is the message bandwidth. Evaluate this improvement for the case when 
W = 15 kHz and f 0 = 2.1 kHz, and compare your result with the corresponding value 
for an FM system. 

Computer Experiments 

2.60 In this experiment we study the behavior of the envelope detector shown in Figure P2.5 
for the following specifications: 

Source resistance, R s = 75 fl 

Load resistance, R; = 10 kfl 

Capacitance, C = 0.01/iF 
The diode has a resistance of 25 fl when it is forward-biased and infinite resistance when 
reverse-biased. 

Compute the waveform of the envelope detector output, assuming an input sinu- 
soidal AM wave with 50 percent modulation. The modulation frequency is 1 kHz, and 
the carrier frequency is 20 kHz. 

2.61 In this experiment we continue the study of the phase-locked loop considered in Section 
2.14: 

( a) Compute variations in the instantaneous frequency of the voltage-controlled oscillator 
m the loop for the following loop parameters: 

Loop-gain parameter, K 0 = — Hz 
1 

Natural frequency, /„ = — Hz 

Damping factor, I = 0.707 
Perform the computations for the following values of frequency step: A/ = 0.125, 
0.5,^, \ Hz. 

(b) For the parameters of the phase-locked loop as specified in Experiment 2 in Section 
2.14, compute how variations in the relative frequency deviation A/ • f m /fl affect 
the peak phase error of the phase-locked loop. 
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fhis chapter, representing the transition from analog to digital communications, covers 
the following topics: 

p, Sampling? which is basic to all forms of pulse modulation. 

p. pulse-amplitude modulation, which is the simplest form of pulse modulation. 

p. Quantization, which, when combined with sampling, permits the representation of an 
analog signal in discrete form in both amplitude and time. 

p- Pulse-code modulation, which is the standard method for the transmission of an analog 
message signal by digital means. 

► Time-division multiplexing, which provides for the time sharing of a common channel by 
a plurality of users by means of pulse modulation. 

► Digital multiplexers, which combine many slow bit streams into a single faster stream. 

► Other forms of digital pulse modulation, namely, delta modulation and differential pulse- 
code modulation. 

► Linear prediction, which is basic to the encoding of analog message signals at reduced bit 
rates as in differential pulse-code modulation. 

► Adaptive forms of differential pulse-code modulation and delta modulation. 

► The MPEG-l/audio coding standard, which is a transparent, perceptually lossless 
compression system. 

13 . 1 Introduction 

In continuous-wave (CW) modulation, which we studied in Chapter 2, some parameter 
of a sinusoidal carrier wave is varied continuously in accordance with the message signal. 
This is in direct contrast to pulse modulation, which we study in the present chapter. In 
pulse modulation, some parameter of a pulse train is varied in accordance with the message 
signal. We may distinguish two families of pulse modulation: analog pulse modulation 
and digital pulse modulation. In analog pulse modulation, a periodic pulse train is used 
as the carrier wave, and some characteristic feature of each pulse (e.g., amplitude, duration, 
or position) is varied in a continuous manner in accordance with the corresponding sample 
value of the message signal. Thus in analog pulse modulation, information is transmitted 
basically in analog form, but the transmission takes place at discrete times. In digital pulse 
modulation, on the other hand, the message signal is represented in a form that is discrete 
in both time and amplitude, thereby permitting its transmission in digital form as a se- 
quence of coded pulses; this form of signal transmission has no CW counterpart. 
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The use of coded pulses for the transmission of analog information-bearing sign a ] s 
represents a basic ingredient in the application of digital communications. This chapter 
may therefore be viewed as a transition from analog to digital communications in oiy 
study of the principles of communication systems. We begin the discussion by describing 
the sampling process, which is basic to all pulse modulation systems, whether they are 
analog or digital. 



3.2 Sampling Process 

The sampling process is usually described in the time domain. As such, it is an operation 
that is basic to digital signal processing and digital communications. Through use of the 
sampling process, an analog signal is converted into a corresponding sequence of samples 
that are usually spaced uniformly in time. Clearly, for such a procedure to have practical 
utility, it is necessary that we choose the sampling rate properly, so that the sequence of 
samples uniquely defines the original analog signal. This is the essence of the sampling 
theorem, which is derived in what follows. 

Consider an arbitrary signal g(t) of finite energy, which is specified for all time. A 
segment of the signal g(t) is shown in Figure 3.1a. Suppose that we sample the signal g(t) 
instantaneously and at a uniform rate, once every T s seconds. Consequently, we obtain an 
infinite sequence of samples spaced T s seconds apart and denoted by {g(«T s )}, where n 
takes on all possible integer values. We refer to T s as the sampling period, and to its 
reciprocal f s = 1IT S as the sampling rate. This ideal form of sampling is called instantaneous 
sampling. 

Let g s (t) denote the signal obtained by individually weighting the elements of a pe- 
riodic sequence of delta functions spaced T s seconds apart by the sequence of numbers 
{g(nT s )}, as shown by (see Figure 3.1b) 

gdt)= E g(nT.) 8{t - nT.) (3.1) 

n= — ™ 

We refer to g B (t) as the ideal sampled signal. The term S(t — nT s ) represents a delta function 
positioned at time t = nT s . From the definition of the delta function, we recall that such 
an idealized function has unit area; see Appendix 2. We may therefore view the multiplying 
factor g{nT s ) in Equation (3.1) as a "mass" assigned to the delta function d(t — nT s ). A 
delta function weighted in this manner is closely approximated by a rectangular pulse of 




Figure 3.1 The sampling process, (a) Analog signal, {b) Instantaneously sampled version of the 
analog signal. 
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duration At and amplitude g(nTJ/At; the smaller we make At the better will be the 
approximation. 

Using the table of Fourier-transform pairs, we may write (see the last item of Table 

A6.3) 

ftW^/. S G(f-mf.) (3.2) 

where G(f) is the Fourier transform of the original signal g(t), and f s is the sampling rate. 
Equation (3.2) states that the process of uniformly sampling a continuous-time signal of 
finite energy results in a periodic spectrum with a period equal to the sampling rate. 

Another useful expression for the Fourier transform of the ideal sampled signal g s (t) 
may be obtained by taking the Fourier transform of both sides of Equation (3.1) and noting 
that the Fourier transform of the delta function 8(t — nT s ) is equal to exp(-j2vnfT s ). Let 
G 6 {f) denote the Fourier transform of gs{t). We may therefore write 

G s (f) = jt g(nT s ) exp(-/2™/TJ (3.3) 

This relation is called the discrete-time Fourier transform. It may be viewed as a complex 
Fourier series representation of the periodic frequency function G s [f), with the sequence 
of samples \g(nT s )} defining the coefficients of the expansion. 

The relations, as derived here, apply to any continuous-time signal g(t) of finite 
energy and infinite duration. Suppose, however, that the signal g(t) is strictly band-limited, 
with no frequency components higher than W Hertz. That is, the Fourier transform G(f) 
of the signal g(t) has the property that G{f) is zero for | f\ a W, as illustrated in Figure 
3.2a; the shape of the spectrum shown in this figure is intended for the purpose of illus- 
tration only. Suppose also that we choose the sampling period T s = 1/2W. Then the 
corresponding spectrum G s (f) of the sampled signal g s (J) is as shown in Figure 3.2b. 
Putting T e = l/2Win Equation (3.3) yields 

CW) = JjW) exp(-^) (3.4) 

From Equation (3.2), we readily see that the Fourier transform of g s (t) may also be 
expressed as 

G 6 (/) = f,G(f) + /, 2 G(f - mf s ) (3.5) 

OT*0 



G(/) 
G(0) 



0 
(a) 




Figure 3.2 (a) Spectrum of a strictly band-limited signalg(t). (b) Spectrum of the sampled 
version of g{t) for a sampling period T s — 1/2W. 
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Hence, under the following two conditions: 

1. G(/) = 0for|/|& W 

2. f, = 2W 

we find from Equation (3.5) that 



G(f) = ^G B (f), -W</<W (3.6) 
Substituting Equation (3.4) into (3.6), we may also write 

^)-^i.»(^)^(-¥> - w<f<w (3 - 7! 

Therefore, if the sample values g(nl2W) of a signal g(t) are specified for all n, then the 
Fourier transform G(f) of the signal is uniquely determined by using the discrete-time 
Fourier transform of Equation (3.7). Because g(t) is related to G{f) by the inverse Fourier 
transform, it follows that the signal g{t) is itself uniquely determined by the sample values 
g(n/2W) for -°° < n< °°. In other words, the sequence {g(n/2W)) has all the information 
contained in g{t). 

Consider next the problem of reconstructing the signal g(t) from the sequence of 
sample values {g(n/2W)}. Substituting Equation (3.7) in the formula for the inverse Fourier 
transform defining g(t) in terms of G(f), we get 

g(t) = \_ G(f) exp(;'27r/f) df 

-Lhk^X-'^-)^" ,ul 

Interchanging the order of summation and integration: 



gW . JL g \2w) 2W 1 



\2Wj 2WJ-w £XP 



df (3.8) 



The integral term in Equation (3.8) is readily evaluated, yielding the final result 
" / n \ sin(2i7 Wf - nir) 
g{t) " Ji. g \2W) (27rWr -~k^T ^ 

= S g{^j sinc(2W* - »), -oo<r<« 

Equation (3.9) provides an interpolation formula for reconstructing the original signal g(t) 
from the sequence of sample values {g(n/2W)}, with the sine function sinc(2Wf) playing 
the role of an interpolation function. Each sample is multiplied by a delayed version of 
the interpolation function, and all the resulting waveforms are added to obtain g(t). 

We may now state the sampling theorem for stricdy band-limited signals of finite 
energy in two equivalent parts, which apply to the transmitter and receiver of a pulse- 
modulation system, respectively: 

1. A band-limited signal of finite energy, which has no frequency components higher 
than W Hertz, is completely described by specifying the values of the signal at instants 
of time separated by 1I2W seconds. 

2. A band-limited signal of finite energy, which has no frequency components higher 
than W Hertz, may be completely recovered from a knowledge of its samples taken 
at the rate of2W samples per second. 
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Figure 3.3 (a) Spectrum of a signal, (b) Spectrum of an undersampled version of the signal 
exhibiting the aliasing phenomenon. 

The sampling rate of 2 W samples per second, for a signal bandwidth of W Hertz, is called 
the Nyquist rate; its reciprocal 1/2W (measured in seconds) is called the Nyquist interval. 

The derivation of the sampling theorem, as described herein, is based on the as- 
sumption that the signal g{t) is strictly band limited. In practice, however, an information- 
bearing signal is not strictly band limited, with the result that some degree of undersam- 
pling is encountered. Consequently, some aliasing is produced by the sampling process. 
Aliasing refers to the phenomenon of a high-frequency component in the spectrum of the 
signal seemingly taking on the identity of a lower frequency in the spectrum of its sampled 
version, as illustrated in Figure 3.3, The aliased spectrum, shown by the solid curve in 
Figure 3.3b, pertains to an "undersampled" version of the message signal represented by 
the spectrum of Figure 3.3a. 

To combat the effects of aliasing in practice, we may use two corrective measures, 
as described here: 

1. Prior to sampling, a low-pass anti-aliasing filter is used to attenuate those high- 
frequency components of the signal that are not essential to the information being 
conveyed by the signal. 

2. The filtered signal is sampled at a rate slightly higher than the Nyquist rate. 

The use of a sampling rate higher than the Nyquist rate also has the beneficial effect of 
easing the design of the reconstruction filter used to recover the original signal from its 
sampled version. Consider the example of a message signal that has been anti-alias (low- 
pass) filtered, resulting in the spectrum shown in Figure 3.4a. The corresponding spectrum 
of the instantaneously sampled version of the signal is shown in Figure 3.4b, assuming a 
sampling rate higher than the Nyquist rate. According to Figure 3.4b, we readily see that 
the design of the reconstruction filter may be specified as follows (see Figure 3.4c): 

*■ The reconstruction filter is low-pass with a passband extending from - W to W, 

which is itself determined by the anti-aliasing filter. 
** The filter has a transition band extending (for positive frequencies) from W to f s - W, 

where f s is the sampling rate. 
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Figure 3.4 (a) Anti-alias altered spectrum of an information-bearing signal, (fc) Spectrum of 
instantaneously sampled version of the signal, assuming the use of a sampling rate greater than 
the Nyquist rate, (c) Magnitude response of reconstruction filter. 

The fact that the reconstruction filter has a well-defined transition band means that it is 
physically realizable. 



1 33 Pulse-Amplitude Modulation 



Now that we understand the essence of the sampling process, we are ready to formally 
define pulse-amplitude modulation, which is the simplest and most basic form of analog 
pulse modulation. In pulse-amplitude modulation (PAM), the amplitudes of regularly 
spaced pulses are varied in proportion to the corresponding sample values of a continuous 
message signal; the pulses can be of a rectangular form or some other appropriate shape. 
Pulse-amplitude modulation as defined here is somewhat similar to natural sampling, 
where the message signal is multiplied by a periodic train of rectangular pulses. However, 
in natural sampling the top of each modulated rectangular pulse varies with the message 
signal, whereas in PAM it is maintained flat; natural sampling is explored further in Prot>- 

km 3 The waveform of a PAM signal is illustrated in Figure 3.5. The dashed curve in this 
figure depicts the waveform of a message signal m(f), and the sequence of amplitude 
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Figure 3.5 Flat-top samples, representing an analog signal. 



modulated rectangular pulses shown as solid lines represents the corresponding PAM sig- 
nal s(t). There are two operations involved in the generation of the PAM signal; 

1. Instantaneous sampling of the message signal m(t) every T s seconds, where the sam- 
pling rate f s = 1/T, is chosen in accordance with the sampling theorem. 

2. Lengthening the duration of each sample so obtained to some constant value T. 

In digital circuit technology, these two operations are jointly referred to as "sample and 
hold." One important reason for intentionally lengthening the duration of each sample is 
to avoid the use of an excessive channel bandwidth, since bandwidth is inversely propor- 
tional to pulse duration. However, care has to be exercised in how long we make the 
sample duration T, as the following analysis reveals. 

Let s(t) denote the sequence of flat-top pulses generated in the manner described in 
Figure 3.5. We may express the PAM signal as 

s(f) = 2 m(nT s )h(t - nT s ) (3.10) 

where T s is the sampling period and m(nT s ) is the sample value of m(t) obtained at time 
t = nT s . The h(t) is a standard rectangular pulse of unit amplitude and duration T, defined 
as follows (see Figure 3.6a): 

f 1, 0 < t < T 
h(t) = \i t = 0,t = T (3.11) 
L0, otherwise 

By definition, the instantaneously sampled version of m(t) is given by 

%W = 2 MnT s ) S(t- nT s ) (3.12) 

where 8(t - nT s ) is a time-shifted delta function. Therefore, convolving m s (t) with the 
pulse h(t), we get 



nT s )h(t - r) dr (3.13) 
nT s )h(t - T)dr 



m s (t) * h(t) = j_ m s (r)h(t - r) dr 
= j_ % MnT s ) 8(t - 
= E m(nT s ) I S(r - 
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Using the sifting property of the delta function (see Appendix 2), we thus obtain 

tnjf) ★ h(t) = E m{nTMt-nT s ) (3.14) 

n— — oo 

From Equations (3.10) and (3.14) it follows that the PAM signal s(t) is mathematically 
equivalent to the convolution of m s (t), the instantaneously sampled version of m(t), and 
the pulse h(t), as shown by 

s(t) = m s (t) ★ h(t) < 3 - 15 ' 
Taking the Fourier transform of both sides of Equation (3.15) and recognizing that 
the convolution of two time functions is transformed into the multiplication of their re- 
spective Fourier transforms, we get 

S(f) = MJJ)H(f) < 3 - l6) 

where S{f) = F[s(t)], W) = FWOl and H{f) = F[h(t)] ^^^fZvouZ 
the problem at hand, we note that the Fourier transform M 5 (/) is related to the Founs 
transform M(/) of the original message signal m{t) as follows: 

M s (f) = fs 2 M(f-kf s ) (3-171 
k — » 

where f s is the sampling rate. Therefore, substitution of Equation (3.17) into (3.16) yields 

S(f) = f, X M[f-kf.)H[f) < 3 - l8! 

k— «. 

Given a PAM signal s(t) whose Fourier transform S(/) is as defined in Equation 
(3.18), how do we recover the original message signal «(*)? As a first step in this reco 
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Figure 3.7 System for recovering message signal m(t) from PAM signal s(t). 



struction, we may pass s(t) through a low-pass filter whose frequency response is defined 
in Figure 3.4c; here it is assumed that the message is limited to bandwidth W and the 
sampling rate f s is larger than the Nyquist rate 2W. Then, from Equation (3.18) we find 
that the spectrum of the resulting filter output is equal to M(f)H(f). This output is equiv- 
alent to passing the original message signal m(t) through another low-pass filter of fre- 
quency response H(f). 

From Equation (3.11) we note that the Fourier transform of the rectangular pulse 
h(t) is given by 

H(/) = T sinc(/T) exp(-/7r/T) (3.19) 

which is plotted in Figure 3.6b. We see therefore that by using flat-top samples to generate 
a PAM signal, we have introduced amplitude distortion as well as a delay of T/2. This 
effect is rather similar to the variation in transmission with frequency that is caused by the 
finite size of the scanning aperture in television. Accordingly, the distortion caused by the 
use of pulse-amplitude modulation to transmit an analog information-bearing signal is 
referred to as the aperture effect. 

This distortion may be corrected by connecting an equalizer in cascade with the low- 
pass reconstruction filter, as shown in Figure 3.7. The equalizer has the effect of decreasing 
the in-band loss of the reconstruction filter as the frequency increases in such a manner as 
to compensate for the aperture effect. Ideally, the magnitude response of the equalizer is 
given by 



|H(/)| Tsinc(/T) sindr/T) 

The amount of equalization needed in practice is usually small. Indeed, for a duty cycle 
TIT, s, 0.1, the amplitude distortion is less than 0.5 percent, in which case the need for 
equalization may be omitted altogether. 

The transmission of a PAM signal imposes rather stringent requirements on the mag- 
nitude and phase responses of the channel, because of the relatively short duration of the 
transmitted pulses. Furthermore, the noise performance of a PAM system can never be 
better than baseband-signal transmission. Accordingly, we find that for transmission over 
long distances, PAM would be used only as a means of message processing for time- 
division multiplexing, from which conversion to some other form of pulse modulation is 
subsequently made; time-division multiplexing is discussed in Section 3.9. ' 



Other Forms of Pulse Modulation 

In a pulse modulation system we may use the increased bandwidth consumed by the pulses 
to improve the noise performance of the system. This can be achieved by representing the 
sample values of the message signals by some property of the pulse other than amplitude: 

*■ Pulse-duration modulation (PDM), also referred to as pulse-width modulation, 
where samples of the message signal are used to vary the duration of the individual 
pulses in the carrier. 
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► Pulse-position modulation (PPM), where the position of a pulse relative to its u n - 
modulated time of occurrence is varied in accordance with the message signal. 

These two other forms of pulse modulation are illustrated in Figure 3.8 for the case of a 

sinusoidal modulating wave. 

In PDM, long pulses expend considerable power while bearing no additional infer- 

mation. If this unused power is subtracted from PDM so that only time transitions are 

preserved, we obtain PPM. Accordingly, PPM is a more efficient form of pulse modulation 

than PDM. . . 

Since in a PPM system the transmitted information is contained in the relative po- 
sitions of the modulated pulses, the presence of additive noise affects the performance of 
such a system by falsifying the time at which the modulated pulses are judged to occur. 
Immunity to noise can be established by making the pulse build up so rapidly that the 
time interval during which noise can exert any perturbation is very short. Indeed, additive 
noise would have no effect on the pulse positions if the received pulses were perfectly 
rectangular, because the presence of noise introduces only vertical perturbations. However, 
the reception of perfectly rectangular pulses would require an infinite channel bandwidth, 
which is of course impractical. Thus with a finite channel bandwidth in practice, we find 
that the received pulses have a finite rise time, so the performance of the PPM receiver is 
affected by noise, which is to be expected. 





junnruuLMi 



(c) 



W) 

FIGURE 3.8 Illustrating two different forms of pulse-time modulation for the case of a sinusoi- 
dal modulating wave. («) Modulating wave, (b) Pulse carrier, (c) PDM wave. (A) PPM wave. 
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As in a CW modulation system, the noise performance of a PPM system may be 
described in terms of the output signal-to-noise ratio (SNR). Also, to find the noise im- 
provement produced by PPM over baseband transmission of a message signal, we may use 
the figure of merit denned as the output signal-to-noise ratio of the PPM system divided 
by the channel signal-to-noise ratio; see Section 2.10. Assuming that the average power of 
the channel noise is small compared to the peak pulse power, the figure of merit of the 
PPM system is proportional to the square of the transmission bandwidth B T normalized 
with respect to the message bandwidth W. When, however, the input signal-to-noise ratio 
drops below a critical value, the system suffers a loss of the wanted message signal at the 
receiver output. That is, a PPM system suffers from a threshold effect of its own. 



3.5 Bandwidth— Noise Trade-Off 

In the context of noise performance, a PPM system is the optimum form of analog pulse 
modulation. The noise analysis of a PPM system reveals that pulse-position modulation 
(PPM) and frequency modulation (FM) systems exhibit a similar noise performance, as 
summarized here. 1 

1. Both systems have a figure of merit proportional to the square of the transmission 
bandwidth normalized with respect to the message bandwidth. 

2. Both systems exhibit a threshold effect as the signal-to-noise ratio is reduced. 

The practical implication of point 1 is that, in terms of a trade-off of increased transmission 
bandwidth for improved noise performance, the best that we can do with continuous-wave 
(CW) modulation and analog pulse modulation systems is to follow a square law. A ques- 
tion that arises at this point in the discussion is: Can we produce a trade-off better than a 
square law? The answer is an emphatic yes, and digital pulse modulation is the way to do 
it. The use of such a method is a radical departure from CW modulation. 

Specifically, in a basic form of digital pulse modulation known as pulse-code mod- 
ulation (PCM), 2 a message signal is represented in discrete form in both time and ampli- 
tude. This form of signal representation permits the transmission of the message signal as 
a sequence of coded binary pulses. Given such a sequence, the effect of channel noise at 
the receiver output can be reduced to a negligible level simply by making the average power 
of the transmitted binary PCM wave large enough compared to the average power of the 
noise. 

Two fundamental processes are involved in the generation of a binary PCM wave: 
sampling and quantization. The sampling process takes care of the discrete-time represen- 
tation of the message signal; for its proper application, we have to follow the sampling 
theorem described in Section 3.2. The quantization process takes care of the discrete- 
amplitude representation of the message signal; quantization is a new process, the details 
of which are described in the next section. For now it suffices to say that the combined 
use of sampling and quantization permits the transmission of a message signal in coded 
form. This, in turn, makes it possible to realize an exponential law for the bandwidth- 
noise trade-off, which is also demonstrated in the next section. 



|__^6 Quantization Process 3 



A continuous signal, such as voice, has a continuous range of amplitudes and therefore its 
samples have a continuous amplitude range. In other words, within the finite amplitude 
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Figure 3.9 Description of a memoryless quantizer. 



range of the signal, we find an infinite number of amplitude levels. It is not necessary i n 
fact to transmit the exact amplitudes of the samples. Any human sense (the ear or the eye), 
as ultimate receiver, can detect only finite intensity differences. This means that the original 
continuous signal may be approximated by a signal constructed of discrete amplitudes 
selected on a minimum error basis from an available set. The existence of a finite number 
of discrete amplitude levels is a basic condition of pulse-code modulation. Clearly, if we 
assign the discrete amplitude levels with sufficiently close spacing, we may make the ap- 
proximated signal practically indistinguishable from the original continuous signal. 

Amplitude quantization is defined as the process of transforming the sample ampli- 
tude m(nT s ) of a message signal m(t) at time t = nT s into a discrete amplitude v (nT s ) taken 
from a finite set of possible amplitudes. We assume that the quantization process is 
memoryless and instantaneous, which means that the transformation at time t = nT s is 
not affected by earlier or later samples of the message signal. This simple form of scalar 
quantization, though not optimum, is commonly used in practice. 

When dealing with a memoryless quantizer, we may simplify the notation by drop- 
ping the time index. We may thus use the symbol m in place of m(nT s ), as indicated in the 
block diagram of a quantizer shown in Figure 3.9a. Then, as shown in Figure. 3.9b, the 
signal amplitude m is specified by the index k if it lies inside the partition cell 

$ k : {m k < m £ m k+ i), k = 1, 2 I (3.21) 

where L is the total number of amplitude levels used in the quantizer. The discrete ampli- 
tudes m k ,k = 1, 2, . . . , L, at the quantizer input are called decision levels or decision 
thresholds. At the quantizer output, the index k is transformed into an amplitude v k that 
represents all amplitudes of the cell $ k ; the discrete amplitudes v k , k = 1, 2, . . . , L, are 
called representation levels or reconstruction levels, and the spacing between two adjacent 
representation levels is called a quantum or step-size. Thus, the quantizer output v equals 
v k if the input signal sample m belongs to the interval $ k . The mapping (see Figure 3.9a) 

v = g(m) (3.22) 

is the quantizer characteristic, which is a staircase function by definition. 

Quantizers can be of a uniform or nonuniform type. In a uniform quantizer, the 
representation levels are uniformly spaced; otherwise, the quantizer is nonuniform. In this 
section, we consider only uniform quantizers; nonuniform quantizers are considered in 
Section 3.7. The quantizer characteristic can also be of midtread or midrise type. Figure 
3.10a shows the input-output characteristic of a uniform quantizer of the midtread type, 
which is so called because the origin lies in the middle of a tread of the staircaselike graph. 
Figure 3.10fe shows the corresponding input-output characteristic of a uniform quantizer 
of the midrise type, in which the origin lies in the middle of a rising part of the staircaselike 
graph. Note that both the midtread and midrise types of uniform quantizers illustrated m 
Figure 3.10 are symmetric about the origin. 
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Figure 3.10 Two types of quantization: («) midtread and (b) midrise. 



■ Quantization Noise 

The use of quantization introduces an error defined as the difference between the input 
signal m and the output signal v. The error is called quantization noise. Figure 3.11 illus- 
trates a typical variation of the quantization noise as a function of time, assuming the use 
of a uniform quantizer of the midtread type. 

Let the quantizer input m be the sample value of a zero-mean random variable M. 
(If the input has a nonzero mean, we can always remove it by subtracting the mean from 
the input and then adding it back after quantization.) A quantizer g(-) maps the input 
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Figure 3. 1 1 Illustration of the quantization process. (Adapted from Bennett, 1948, with 
permission of AT&T.) 
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random variable M of continuous amplitude into a discrete random variable V; their 
respective sample values m and v are related by Equation (3.22). Let the quantization error 
be denoted by the random variable Q of sample value q. We may thus write 

q = m - v (3.23] 

or, correspondingly, 

Q = M - V (3-24) 

With the input M having zero mean, and the quantizer assumed to be symmetric as i n 
Figure 3 10, it follows that the quantizer output V and therefore the quantization error 
O will also have zero mean. Thus for a partial statistical characterization of the quantizer 
in terms of output signal-to-(quantization) noise ratio, we need only find the mean-square 
value of the quantization error Q. 

Consider then an input m of continuous amplitude in the range ( »w, m ), 
Assuming a uniform quantizer of the midrise type illustrated in Figure 3.10fo, we find that 
the step-size of the quantizer is given by 

A = ^ (3.25) 

where L is the total number of representation levels. For a uniform quantizer the quan- 
tization error Q will have its sample values bounded by -A/2 s q ^ A/2. If the step- sl ze 
A is sufficiently small (i.e., the number of representation levels L is sufficiently large) it is 
reasonable to assume that the quantization error Q is a uniformly distributed random 
variable, and the interfering effect of the quantization noise on the quantizer input is similar 
to that of thermal noise. We may thus express the probability density function of the 
quantization error Q as follows: 

fl A A 

m= a- -2***2 < 3 - 26 > 

[o, otherwise 

For this to be true, however, we must ensure that the incoming signal does not overload 
the quantizer. Then, with the mean of the quantization error being zero, its variance o* Q 
is the same as the mean-square value: 

<rh = E[Q 2 ] (3.27) 

pA/2 

■? 2 /q(<?) <te 

J -A/2 

Substituting Equation (3.26) into (3.27), we get 



1 f 

u A J -a/2 



-A/2 

a 2 da 

(3.28) 



= ^ 
12 

Typically, the L-ary number k, denoting the feth representation level of the quanti^. 
is transmitted to the receiver in binary form. Let R denote the number of bits per samp 
used in the construction of the binary code. We may then write 

L = 2 * (3.2?) 
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or, equivalently, 

R = log 2 L (3.30) 
Hence, substituting Equation (3.29) into (3.25), we get the step size 

A = ^= (3.31) 

Thus the use of Equation (3.31) in (3.28) yields 

a% = imLx2- 2R (3.32) 

Let P denote the average power of the message signal m(t). We may then express the output 
signal-to-noise ratio of a uniform quantizer as 



(SNR) 0 = -J 

°Q 



3P 

mL,, 



(3.33) 

2^r 



Equation (3.33) shows that the output signal-to-noise ratio of the quantizer increases ex- 
ponentially with increasing number of bits per sample, R. Recognizing that an increase in 
R requires a proportionate increase in the channel (transmission) bandwidth B T , we thus 
see that the use of a binary code for the representation of a message signal (as in pulse- 
code modulation) provides a more efficient method than either frequency modulation (EM) 
or pulse-position modulation (PPM) for the trade-off of increased channel bandwidth for 
improved noise performance. In making this statement, we presume that the FM and PPM 
systems are limited by receiver noise, whereas the binary-coded modulation system is lim- 
ited by quantization noise. We have more to say on the latter issue in Section 3.8. 



*>■ Example 3.1 Sinusoidal Modulating Signal 

Consider the special case of a full-load sinusoidal modulating signal of amplitude A m , which 
utilizes all the representation levels provided. The average signal power is (assuming a load 
of 1 ohm) 

The total range of the quantizer input is 2A m , because the modulating signal swings between 
— A m and A„. We may therefore set m m!i = A„, in which case the use of Equation (3.32) 
yields the average power (variance) of the quantization noise as 

Thus the output signal-to-noise ratio of a uniform quantizer, for a full-ioad test tone, is 

(SNR)o = = f (2«) (3.34) 

Expressing the signal-to-noise ratio in decibels, we get 

10 log 10 (SNR) o = 1.8 + 6R (3.35) 
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TABLE 3.1 Signal-to-(quantization) noise ratio 
, for varying number of re-presentation levels 
\ for sinusoidal modulation 



64 



256 



Number of 

Number of Representation Bits per Signal-to-Noise 

Levels, L Sample, R Ratio (dB) 

32 5 3L8 



6 37.8 



128 7 43.8 



49.8 



For various values of L and R, the corresponding values of signal-to-noise ratio are as given 
in Table 3.1. From Table 3.1 we can make a quick estimate of the number of bits per sample 
required for a desired output signal-to-noise ratio, assuming sinusoidal modulation. « 

Thus far in this section we have focused on how to characterize memoryless scalar 
quantizers and assess their performance. In so doing, however, we avoided the optimum 
design of quantizers, that is, the issue of selecting the representation levels and partition 
cells so as to minimize the average quantization power for a prescribed number of repre- 
sentation levels. Unfortunately, this optimization problem does not lend itself to a closed- 
form solution because of the highly nonlinear nature of the quantization process. Rather, 
we have effective algorithms for finding the optimum design in an iterative manner. A well- 
known algorithm that deserves to be mentioned in this context is the Lloyd-Max quantizer, 
which is discussed next. 

e Conditions for O ptimalcty of Scalar Quantizers 

In designing a scalar quantizer the challenge is how to select the representation levels and 
surrounding partition cells so as to minimize the average quantization power for a fixed 
number of representation levels. 

To state the problem in mathematical terms, consider a message signal m(t) drawn 
from a stationary process M(t). Let -A <»> =£ A denote the dynamic range of \m(t), which 
is partitioned into a set of L cells, as depicted in Figure 3.12. The boundaries of the 
partition cells are defined by a set of real numbers m u m 2 , . . . , m L+1 that satisfy the 
following three conditions: 

m 1 = -A 
m L+1 = A 

m k £ ntk+i f° r k = 1, 2,. . . , L 

The kth partition cell is denned by 

3> k : m k <m < m k +, for k = 1, 2, . . . , L 



(3.36) 



mi = -A 

H — 



J 



Figure 3.12 Illustrating the partitioning of the dynamic range -AsuSAofa message 
signal m(i) into a set of L cells. 
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Let the representation levels (i.e., quantization values) be denoted by v k , k = 1, 2, . . . , L. 
Then, assuming that d(m, Vk) denotes a distortion measure for using v± to represent all 
those values of the input m that He inside the partition cell 3> k , the goal is to find the two 
sets, {VkYk=\ and {$>k]k-i, that minimize the average distortion 

D = 2 f d(m, v k )f M (m) dm (3.37) 

where f M (m) is the probability density function of the random variable M with sample 
value m. 

A commonly used distortion measure is 

d(m, v k ) = (m - v k f (3.38) 

in which case we speak of the mean-square distortion. In any event, the optimization 
problem stated herein is nonlinear, defying an explicit, closed-form solution. To get around 
this difficulty, we resort to an algorithmic approach for solving the problem in an iterative 
manner. 

Structurally speaking, the quantizer consists of two components with interrelated 
design parameters: 

► An encoder characterized by the set of partition cells {$ k }k-ii it is located in the 
transmitter. 

► A decoder characterized by the set of representation levels {v k }% =1 ; it is located in the 
receiver. 

Accordingly, we may identify two critically important conditions that provide the math- 
ematical basis for all algorithmic solutions to the optimum quantization problem. One 
condition assumes that we are given a decoder and the problem is to find the optimum 
encoder in the transmitter. The other condition assumes that we are given an encoder and 
the problem is to find the optimum decoder in the receiver. Henceforth, these two condi- 
tions are referred to as condition I and condition II, respectively. 



Condition I. Optimality of the Encoder for a Given Decoder 



The availability of a decoder means that we have a certain codebook in mind. Let the 
codebook be defined by 

«:fa}£-i (3.39) 

Given the codebook the problem is to find the set of partition cells {S-k)k=i that mini- 
mizes the average distortion D. That is, we wish to find the encoder defined by the non- 
linear mapping 

g(m) = v k , k = 1, 2, . . . , L (3.40) 

such that we have 

D = f d(m, g(m))f M (m) <fM > £ f [min d(m, v b )]f M (m) dm (3.41) 

For the lower bound specified in Equation (3.41) to be attained, we require that the non- 
linear mapping of Equation (3.40) be satisfied only if the condition 

d{m, v k ) & d(m, v,) holds for all / # k (3.42) 



200 Chapter 3 ■ Pulse Modulation 



The necessary condition described in Equation (3.42) for optimality of the encoder for a 
specified codebook <« is recognized as the nearest neighbor condition. In words, the nearest 
neighbor condition requires that the partition cell » k should embody all those values 0 f 
the input m that are closer to v k than any other element of the codebook C. This optimahty 
condition is indeed intuitively satisfying. 



Conditio n II. Optimally of the Decoder fo r a Given Encoder 

Consider next the reverse situation to that described under condition I, which may be 
stated as follows: Optimize the codebook <€ = {«/*}*_, for the decoder, given that the set 
of partition cells [$ k }U characterizing the encoder is fixed. The criterion for optimization 
is the average (mean-square) distortion: 

D = £ [ (m- v k ff M (m) dm (3.43) 

The probability density function f M (m) is clearly independent of the codebook <6. Hence, 
differentiating D with respect to the representation level v k , we readily obtain 

*° = -2 i f (m- v k )f M (m) dm (3.44) 

dV k k=l - l ™ej> t 

Setting dD/Sv k equal to zero and then solving for v k , we obtain the optimum value 

mf M (m) dm 

= J J^i (3.45) 

V fe, Opt j- 

f M (m) dm 

The denominator in Equation (3.45) is just the probability, p k , that the random variable 
M with sample value m lies in the partition cell $ k , as shown by 

p k = P(m 4 <M^m k +l) ^ A6) 

/mM dm 

Accordingly, we may interpret the optimality condition of Equation (3.45) as choosing 
the representation level v k to equal the conditional mean of the random variable M, g.ven 
that M lies in the partition cell f h . We can thus formally state the condition for optimahty 
of the decoder for a given encoder as follows: 

^ It =E[M|m s <Msm l+1 ] ( 3 ' 47 > 

where E is the expectation operator. Equation (3.47) is also intuitively satisfying. 

Note that the nearest neighbor condition (condition I) for optimahty of the encoae 
for a given decoder was proved for a generic average distortion. However, the conditions 
mean requirement (condition II) for optimality of the decoder for a given encoder wu 
proved for the special case of a mean-square distortion. In any event, these two condition* 
are necessary for optimality of a scalar quantizer. Basically, the algorithm for design 
the quantizer consists of alternately optimizing the encoder m accordance with conditio 
I then optimizing the decoder in accordance with condition II, and continumg in tn 
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manner until the average distortion D reaches a minimum. An optimum quantizer designed 
in this manner is called a Lloyd-Max quantizer. 4 



Pulse-Code Modulation 



With the sampling and quantization processes at our disposal, we are now ready to de- 
scribe pulse-code modulation, which, as mentioned previously, is the most basic form of 
digital pulse modulation. In pulse-code modulation (PCM), a message signal is represented 
by a sequence of coded pulses, which is accomplished by representing the signal in discrete 
form in both time and amplitude. The basic operations performed in the transmitter of a 
PCM system are sampling, quantizing, and encoding, as shown in Figure 3.13a; the low- 
pass filter prior to sampling is included to prevent aliasing of the message signal. The 
quantizing and encoding operations are usually performed in the same circuit, which is 
called an analog-to-digital converter. The basic operations in the receiver are regeneration 
of impaired signals, decoding, and reconstruction of the train of quantized samples, as 
shown in Figure 3.13c. Regeneration also occurs at intermediate points along the trans- 
mission path as necessary, as indicated in Figure 3.13b. When time-division multiplexing 
is used, it becomes necessary to synchronize the receiver to the transmitter for the overall 
system to operate satisfactorily, as discussed in Section 3.9. In what follows, we describe 
the various operations that constitute a basic PCM system. 



■ Sampling 

The incoming message signal is sampled with a train of narrow rectangular pulses so as 
to closely approximate the instantaneous sampling process. To ensure perfect reconstruc- 
tion of the message signal at the receiver, the sampling rate must be greater than twice the 
highest frequency component W of the message signal in accordance with the sampling 
theorem. In practice, a low-pass anti-aliasing filter is used at the front end of the sampler 
to exclude frequencies greater than W before sampling. Thus the application of sampling 
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FIGURE 3.13 The basic elements of a PCM system. 
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permits the reduction of the continuously varying message signal (of some finite duration) 
to a limited number of discrete values per second. 



a Quantization 

The sampled version of the message signal is then quantized, thereby providing a new 
representation of the signal that is discrete in both time and amplitude. The quantization 
process may follow a uniform law as described in Section 3.6. In telephonic communica- 
tion, however, it is preferable to use a variable separation between the representation levels. 
For example, the range of voltages covered by voice signals, from the peaks of loud talk 
to the weak passages of weak talk, is on the order of 1000 to 1. By using a nonuniform 
quantizer with the feature that the step-size increases as the separation from the origin of 
the input-output amplitude characteristic is increased, the large end steps of the quantizer 
can take care of possible excursions of the voice signal into the large amplitude ranges 
that occur relatively infrequently. In other words, the weak passages, which need more 
protection, are favored at the expense of the loud passages. In this way, a nearly uniform 
percentage precision is achieved throughout the greater part of the amplitude range of the 
input signal, with the result that fewer steps are needed than would be the case if a uniform 
quantizer were used. 

The use of a nonuniform quantizer is equivalent to passing the baseband signal 
through a compressor and then applying the compressed signal to a uniform quantizer. A 
particular form of compression law that is used in practice is the so-called p,-law, 5 which 
is defined by 

log(l + M H) (3 . 48) 

where m and v are the normalized input and output voltages, and p is a positive constant. 
In Figure 3.14fl, we have plotted the /u-law for three different values of p. The case of 
uniform quantization corresponds to p = 0. For a given value of the reciprocal slope 




Figure 3.14 Compression laws, (a) /i-law. (b) A-law. 
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of the compression curve, which defines the quantum steps, is given by the derivative of 
| m | with respect to | v \ ; that is, 

d\m\ _ log{l +fi) 



t 1 



(1 + fi\m\) 



(3.49) 



We see therefore that the /i-law is neither strictly linear not strictly logarithmic, but it is 
approximately linear at low input levels corresponding to ju. | m « 1, and approximately 
logarithmic at high input levels corresponding to n | m | 5>> 1. 

Another compression law that is used in practice is the so-called A-law defined by 



A\m\ 



log A' 
log(A|?w|) 



1 + log A 



0- 
1 

< 

A 



1 

^ — 
A 



(3.50) 



which is plotted in Figure 3.14fr for varying A. The case of uniform quantization corre- 
sponds to A — 1. The reciprocal slope of this second compression curve is given by 



the derivative of | m \ with respect to 
to the normalized input | m \ ) 

"l 

d\ 



v I , as shown by (depending on the value assigned 



log A 
A 



(1 + A)\m\ 



0 i: m - 
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— < \m \ ' 



(3.51) 



To restore the signal samples to their correct relative level, we must, of course, use 
a device in the receiver with a characteristic complementary to the compressor. Such a 
device is called an expander. Ideally, the compression and expansion laws are exactly 
inverse so that, except for the effect of quantization, the expander output is equal to the 
compressor input. The combination of a compressor and an expander is called a 
compander. 

For both the /x-law and A-law, the dynamic range capability of the compander im- 
proves with increasing fx and A, respectively. The SNR for low-level signals increases at 
the expense of the SNR for high-level signals. To accommodate these two conflicting 
requirements (i.e., a reasonable SNR for both low- and high-level signals), a compromise 
is usually made in choosing the value of paramenter fx for the /i-law and parameter A for 
the A-law. The typical values used in practice are: fx = 255 and A = 87.6. 

It is also of interest to note that in actual PCM systems, the companding circuitry 
does not produce an exact replica of the nonlinear compression curves shown in Figure 
3.14. Rather, it provides a piecewise linear approximation to the desired curve. By using 
a large enough number of linear segments, the approximation can approach the true com- 
pression curve very closely. This form of approximation is illustrated in Example 3.2. 



■ Encoding 

In combining the processes of sampling and quantization, the specification of a continuous 
message (baseband) signal becomes limited to a discrete set of values, but not in the form 
best suited to transmission over a telephone line or radio path. To exploit the advantages 
of sampling and quantizing for the purpose of making the transmitted signal more robust 
to noise, interference and other channel impairments, we require the use of an encoding 
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I TABLE 3.2 Binary number system 
for R = 4 bits/sample 

Ordinal Number of Level Number Expressed as Binary 
Representation Level Sum of Powers of 2 Number 



0 




0000 


1 


2° 


0001 


2 


2 1 


0010 


3 


2 1 + 2° 


0011 


4 


2 2 


0100 


5 


2 2 + 2° 


0101 


6 


2 2 + 2 1 


0110 


7 


2 2 + 2 1 + 2° 


0111 


8 


2 3 


1000 


9 


2 3 +2° 


1001 


10 


2 3 +2 1 


1010 


11 


2 3 + 2 1 + 2° 


1011 


12 


2 3 + 2 2 


1100 


13 


2 3 + 2 2 + 2° 


1101 


14 


2 3 + 2 2 + 2 1 


1110 


15 


2 3 + 2 2 + 2 1 + 2° 


1111 



process to translate the discrete set of sample values to a more appropriate form of signal. 
Any plan for representing each of this discrete set of values as a particular arrangement of 
discrete events is called a code. One of the discrete events in a code is called a code element 
or symbol. For example, the presence or absence of a pulse is a symbol. A particular 
arrangement of symbols used in a code to represent a single value of the discrete set is 
called a code word or character. 

In a binary code, each symbol may be either of two distinct values or kinds, such as 
the presence or absence of a pulse. The two symbols of a binary code are customarily 
denoted as 0 and 1. In a ternary code, each symbol may be one of three distinct values or 
kinds, and so on for other codes. However, the maximum advantage over the effects of 
noise in a transmission medium is obtained by using a binary code, because a binary 
symbol withstands a relatively high level of noise and is easy to regenerate. Suppose that, 
in a binary code, each code word consists of R bits: bit is an acronym for binary digit; 
thus R denotes the number of bits per sample. Then, using such a code, we may represent 
a total of 2 R distinct numbers. For example, a sample quantized into one of 256 levels 
may be represented by an 8-bit code word. 

There are several ways of establishing a one-to-one correspondence between repre- 
sentation levels and code words. A convenient method is to express the ordinal number 
of the representation level as a binary number. In the binary number system, each digit 
has a place-value that is a power of 2, as illustrated in Table 3.2 for the case of four bits 
per sample (i.e., -R = 4). 

Line Codes 

Any of several line codes can be used for the electrical representation of a binary 
data stream. Figure 3.15 displays the waveforms of five important line codes for the ex- 
ample data stream 01101001. Figure 3.16 displays their individual power spectra (tor 
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Figure 3.15 Line codes for the electrical representations of binary data, (a) Unipolar NRZ 
signaling, (b) Polar NRZ signaling, (c) Unipolar RZ signaling, (d) Bipolar RZ signaling. 
(e) Split-phase or Manchester code. 



positive frequencies) for randomly generated binary data, assuming that (1) symbols 0 and 
1 are equiprobable, (2) the average power is normalized to unity, and (3) the frequency/ 
is normalized with respect to the bit rate VT b . (For the formulas used to plot the power 
spectra of Figure 3.16, the reader is referred to Problem 3.11.) The five line codes illustrated 
in Figure 3.15 are described here: 

1. Unipolar nonreturn-to-zero (NRZ) signaling 

In this line code, symbol 1 is represented by transmitting a pulse of amplitude A for the 
duration of the symbol, and symbol 0 is represented by switching off the pulse, as in Figure 
3.15a. This line code is also referred to as on-off signaling. Disadvantages of on-off sig- 
naling are the waste of power due to the transmitted DC level and the fact that the power 
spectrum of the transmitted signal does not approach zero at zero frequency. 

2. Polar nonreturn-to-zero (NRZ) signaling 

In this second line code, symbols 1 and 0 are represented by transmitting pulses of ampli- 
tudes +A and -A, respectively, as illustrated in Figure 3.15fr. This line code is relatively 
easy to generate but its disadvantage is that the power spectrum of the signal is large near 
zero frequency. 
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A 2 = l 




Normalized frequency 



Figure 3.16 Power spectra of line codes: (a) Unipolar NRZ signal, (b) Polar NRZ signal, 
(c) Unipolar RZ signal, (d) Bipolar RZ signal, (e) Manchester-encoded signal. The frequency B 
normalized with respect to the bit rate l/T h , and the average power is normalized to unity. 
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3. Unipolar return-to-zero (RZ) signaling 

In this other line code, symbol 1 is represented by a rectangular pulse of amplitude A and 
half-symbol width, and symbol 0 is represented by transmitting no pulse, as illustrated in 
Figure 3.15c. An attractive feature of this line code is the presence of delta functions at 
/ = 0, ±1/T h in the power spectrum of the transmitted signal, which can be used for bit- 
timing recovery at the receiver. However, its disadvantage is that it requires 3 dB more 
power than polar return-to-zero signaling for the same probability of symbol error; this 
issue is addressed in Chapter 4 under Problem 4.10. 

4. Bipolar return-to-zero (BRZ) signaling 

This line code uses three amplitude levels as indicated in Figure 3. lSd. Specifically, positive 
and negative pulses of equal amplitude (i.e., +A and -A) are used alternately for symbol 
1, with each pulse having a half-symbol width; no pulse is always used for symbol 0. A 
useful property of the BRZ signaling is that the power spectrum of the transmitted signal 
has no DC component and relatively insignificant low-frequency components for the case 
when symbols 1 and 0 occur with equal probability. This line code is also called alternate 
mark inversion (AMI) signaling. 

5. Split-phase (Manchester code) 

In this method of signaling, illustrated in Figure 3.15e, symbol 1 is represented by a positive 
pulse of amplitude A followed by a negative pulse of amplitude -A, with both pulses being 
half-symbol wide. For symbol 0, the polarities of these two pulses are reversed. The Man- 
chester code suppresses the DC component and has relatively insignificant low-frequency 
components, regardless of the signal statistics. This property is essential in some 
applications. 

Differential Encoding 

This method is used to encode information in terms of signal transitions. In partic- 
ular, a transition is used to designate symbol 0 in the incoming binary data stream, while 
no transition is used to designate symbol 1, as illustrated in Figure 3.17. In Figure 3.17fr 
we show the differentially encoded data stream for the example data specified in Figure 
3.17a. The original binary data stream used here is the same as that used in Figure 3.15. 
The waveform of the differentially encoded data is shown in Figure 3.17c, assuming the 
use of unipolar nonreturn-to-zero signaling. From Figure 3.17 it is apparent that a differ- 
entially encoded signal may be inverted without affecting its interpretation. The original 
binary information is recovered simply by comparing the polarity of adjacent binary sym- 
bols to establish whether or not a transition has occurred. Note that differential encoding 
requires the use of a reference bit before initiating the encoding process. In Figure 3.17, 
symbol 1 is used as the reference bit. 

la) Original binary data 01101001 
lb) Differentially encoded data 100011011 



Time — *- 

Figure 3.17 (a) Original binary data, (h) Differentially encoded data, assuming reference bit I. 
(c) Waveform of differentially- encoded data using unipolar NRZ signaling. 



(c) Waveform 
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FIGURE 3.18 Block diagram of regenerative repeater. 



■ Regeneration 

The most important feature of PCM systems lies in the ability to control the effects of 
distortion and noise produced by transmitting a PCM signal through a channel. This 
capability is accomplished by reconstructing the PCM signal by means of a chain of re- 
generative repeaters located at sufficiently close spacing along the transmission route. As 
illustrated in Figure 3.18, three basic functions are performed by a regenerative repeater: 
equalization, timing, and decision making. The equalizer shapes the received pulses so as 
to compensate for the effects of amplitude and phase distortions produced by the nomdeal 
transmission characteristics of the channel. The timing circuitry provides a periodic pulse 
train derived from the received pulses, for sampling the equalized pulses at the instants of 
time where the signal-to-noise ratio is a maximum. Each sample so extracted is compared 
to a predetermined threshold in the decision-making device. In each bit interval, a decision 
is then made whether the received symbol is a 1 or a 0 on the basis of whether the threshold 
is exceeded or not. If the threshold is exceeded, a clean new pulse representmg symbol 
1 is transmitted to the next repeater. Otherwise, another clean new pulse representing 
symbol 0 is transmitted. In this way, the accumulation of distortion and noise in a repeater 
span is completely removed, provided that the disturbance is not too large to cause an 
error in the decision-making process. Ideally, except for delay, the regenerated signal is 
exactly the same as the signal originally transmitted. In practice, however, the regenerated 
signal departs from the original signal for two main reasons: 

1 The unavoidable presence of channel noise and interference causes the repeater to 
make wrong decisions occasionally, thereby introducing bit errors into the regener- 
ated signal. 

2. If the spacing between received pulses deviates from its assigned value, a jitter is 
introduced into the regenerated pulse position, thereby causing distortion. 



■ Decoding 

The first operation in the receiver is to regenerate (i.e., reshape and clean up) the received 
pulses one last time. These clean pulses are then regrouped into code words and decoded 
(i e mapped back) into a quantized PAM signal. The decoding process involves generatffl 
a pulse the amplitude of which is the linear sum of all the pulses in the code word, with 
each pulse being weighted by its place value (2°,2\ 2\ . . . , 2*->) in the code, where R 
the number of bits per sample. 

■ Filtering 

The final operation in the receiver is to recover the message signal by passing the decoder 
output through a low-pass reconstruction filter whose cutoff frequency is equal to tn 
message bandwidth W. Assuming that the transmission path is error free, the recover 
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signal includes no noise with the exception of the initial distortion introduced by the 
quantization process. 

Noise Considerations in PCM Systems 

The performance of a PCM system is influenced by two major sources of noise: 

1. Channel noise, which is introduced anywhere between the transmitter output and 
the receiver input. Channel noise is always present, once the equipment is switched 
on. 

2. Quantization noise, which is introduced in the transmitter and is carried all the way 
along to the receiver output. Unlike channel noise, quantization noise is signal- 
dependent in the sense that it disappears when the message signal is switched off. 

Naturally, these two sources of noise appear simultaneously once the PCM system is in 
operation. However, the traditional practice is to consider them separately, so that we may 
develop insight into their individual effects on the system performance. 

The main effect of channel noise is to introduce bit errors into the received signal. 
In the case of a binary PCM system, the presence of a bit error causes symbol 1 to be 
mistaken for symbol 0, or vice versa. Clearly, the more frequently bit errors occur, the 
more dissimilar the receiver output becomes compared to the original message signal. The 
fidelity of information transmission by PCM in the presence of channel noise may be 
measured in terms of the average probability of symbol error, which is defined as the 
probability that the reconstructed symbol at the receiver output differs from the transmit- 
ted binary symbol, on the average. The average probability of symbol error, also referred 
to as the bit error rate (BER), assumes that all the bits in the original binary wave are of 
equal importance. When, however, there is more interest in reconstructuring the analog 
waveform of the original message signal, different symbol errors may need to be weighted 
differently; for example, an error in the most significant bit in a code word (representing 
a quantized sample of the message signal) is more harmful than an error in the least 
significant bit. 

To optimize system performance in the presence of channel noise, we need to mini- 
mize the average probability of symbol error. For this evaluation, it is customary to model 
the channel noise as additive, white, and Gaussian. The effect of channel noise can be 
made practically negligible by ensuring the use of an adequate signal energy-to-noise den- 
sity ratio through the provision of short-enough spacing between the regenerative repeaters 
in the PCM system. In such a situation, the performance of the PCM system is essentially 
limited by quantization noise acting alone. 

From the discussion of quantization noise presented in Section 3.6, we recognize that 
quantization noise is essentially under the designer's control. It can be made negligibly 
small through the use of an adequate number of representation levels in the quantizer and 
the selection of a companding strategy matched to the characteristics of the type of message 
signal being transmitted. We thus find that the use of PCM offers the possibility of building 
a communication system that is rugged with respect to channel noise on a scale that is 
beyond the capability of any CW modulation or analog pulse modulation system. 

a Error Threshold 

The underlying theory of bit error rate calculation in a PCM system is deferred until 
Chapter 4. For the present, it suffices to say that the average probability of symbol error 
in a binary encoded PCM receiver due to additive white Gaussian noise depends solely on 
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E JNo which is defined as the ratio of the transmitted signal energy per bit E, to the 
noise spearal density, N 0 . Note that the ratio E b IN 0 is dimensions even though the 
cuant tii t and N^have different physical meaning. In Table 3.3 we present a summary 
ofth dependence or the case of a binary PCM system using polar nonreturn-to-zero 
InaLg ^e es U l ts presented in the last column of the table assume a bit rate of 10 b/, 
From Table 3 3 it is clear that there is an error threshold (at about 11 dB). For 
E,/N 0 bebw the error threshold the receiver performance 7° 1 ^.*^ n 2 1 J™ 1 of 
errors and above it the effect of channel noise is practically negligible, bi other words, 
errors ana aooye threshold, channel noise has virtually no 

E£ ^TX^ ^ - ^ the goal of PCM. When however, 
£ N„ drop bdow the error threshold, there is a sharp increase in the rate at wh,ch errors 
o ccur in the receiver. Because decision errors result in the construction of incorrect code 
IXweLTthatwhen the errors are frequent, the reconstructed message at the receiver 
nutrmt bears little resemblance to the original message. 

P ComTa ng the figure of 11 dB for the error threshold in a PCM system using polar 
NRZ sKE "with the 60-70 dB required for high-quality transrmss.on of speech using 
SitTmodulation, we see that PCM requires much less power, even though the av- 
Tage noTseTower in the PCM system is increased by the R-fold increase ,n bandwidth, 
where R is the number of bits in a code word (i.e., bits per sample). 

In most transmission systems, the effects of noise and distortion from the .individual 
links accumulate. For a given quality of overall transmission, the longer the physical sep- 
Laln bew en he transmitter and the receiver, the more severe are the requirements on 
eS linktTthe system. In a PCM system, however, because the signal can be regenerate 
as often^ a necessary, the effects of amplitude, phase, and nonlinear distortions in one link 
S no TtooTvere) have practically no effect on the regenerated input signa to the nex, 
L We Lve a so see „ that the effect of channel noise can be made practically negligible 
link. We have also threshold . For a H prac tical purposes, then, the transmission 

SSS PCMHnkt almost independent of the physical length of the com- 

mUni toThe C r h important characteristic of a PCM system is its ru gg edness to ,nterf„ 
caused by stray impulses or cross-talk. The combined presence of channe noise and - 
erfe end causes the error threshold necessary for satisfactory operation of the VtM w 
tem to increase If an adequate margin over the error threshold is provided in he & t 
ZThowZr'L system can withstand the presence of relatively large amounts of m- 
feSSher words, a PCM system is robust to channel noise and mterference. 

I TABLE 3.3 Influence of E b /N 0 on the 
probability of error 



E„/N 0 



For a Bit Rate of 10 5 bis, 
Probability of This Is About One 

Error P„ Error Ever y 



4.3 dB 10- 2 10 3 second 



8.4 10 
10.6 10 
12.0 
13.0 



10 _1 second 
10 seconds 
j0-8 20 minutes 

10~ 10 1 day 



14Q 10 -i2 3 months 
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3.9 Time-Division Multiplexing 



The sampling theorem provides the basis for transmitting the information contained in a 
band-limited message signal m(t) as a sequence of samples of m[t) taken uniformly at a 
rate that is usually slightly higher than the Nyquist rate. An important feature of the 
sampling process is a conservation of time. That is, the transmission of the message samples 
engages the communication channel for only a fraction of the sampling interval on a 
periodic basis, and in this way some of the time interval between adjacent samples is cleared 
for use by other independent message sources on a time-shared basis. We thereby obtain 
a time-division multiplex (TDM) system, which enables the joint utilization of a common 
communication channel by a plurality of independent message sources without mutual 
interference among them. 

The concept of TDM is illustrated by the block diagram shown in Figure 3.19. Each 
input message signal is first restricted in bandwidth by a low-pass anti-aliasing filter to 
remove the frequencies that are nonessential to an adequate signal representation. The 
low-pass filter outputs are then applied to a commutator, which is usually implemented 
using electronic switching circuitry. The function of the commutator is twofold: (1 ) to take 
a narrow sample of each of the N input messages at a rate f s that is slightly higher than 
2W, where W is the cutoff frequency of the anti-aliasing filter, and (2) to sequentially 
interleave these N samples inside the sampling interval T s . Indeed, this latter function is 
the essence of the time-division multiplexing operation. Following the commutation pro- 
cess, the multiplexed signal is applied to a pulse modulator, the purpose of which is to 
transform the multiplexed signal into a form suitable for transmission over the common 
channel. It is clear that the use of time-division multiplexing introduces a bandwidth ex- 
pansion factor N, because the scheme must squeeze N samples derived from N independent 
message sources into a time slot equal to one sampling interval. At the receiving end of 
the system, the received signal is applied to a pulse demodulator, which performs the 
reverse operation of the pulse modulator. The narrow samples produced at the pulse de- 
modulator output are distributed to the appropriate low-pass reconstruction filters by 
means of a decommutator, which operates in synchronism with the commutator in the 
transmitter. This synchronization is essential for a satisfactory operation of the system. 
The way this synchronization is implemented depends naturally on the method of pulse 
modulation used to transmit the multiplexed sequence of samples. 

The TDM system is highly sensitive to dispersion in the common channel, that is, to 
variations of amplitude with frequency or lack of proportionality of phase with frequency. 
Accordingly, accurate equalization of both magnitude and phase responses of the channel 
is necessary to ensure a satisfactory operation of the system; this issue is discussed in 
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Figure 3.19 Block diagram of TDM system. 
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Chapter 4. However, unlike FDM, to a first-order approximation TDM is immune to 
nonlinearities in the channel as a source of cross-talk. The reason for this behavior is that 
different message signals are not simultaneously applied to the channel. 



■ Synchronization 

In applications using PCM, it is natural to multiplex different messages sources by time 
division, whereby each source keeps its individuality throughout the journey from the 
transmitter to the receiver. This individuality accounts for the comparative ease with which 
message sources may be dropped or reinserted in a time-division multiplex system. As the 
number of independent message sources is increased, the time interval that may be allotted 
to each source has to be reduced, since all of them must be accommodated into a time 
interval equal to the reciprocal of the sampling rate. This, in turn, means that the allowable 
duration of a code word representing a single sample is reduced. However, pulses tend to 
become more difficult to generate and to transmit as their duration is reduced. Further- 
more, if the pulses become too short, impairments in the transmission medium begin to 
interfere with the proper operation of the system. Accordingly, in practice, it is necessary 
to restrict the number of independent message sources that can be included within a time- 
division group. 

In any event, for a PCM system with time-division multiplexing to operate satisfac- 
torily, it is necessary that the timing operations at the receiver, except for the time lost in 
transmission and regenerative repeating, follow closely the corresponding operations at 
the transmitter. In a general way, this amounts to requiring a local clock at the receiver 
to keep the same time as a distant standard clock at the transmitter, except that the local 
clock is somewhat slower by an amount corresponding to the time required to transport 
the message signals from the transmitter to the receiver. One possible procedure to syn- 
chronize the transmitter and receiver clocks is to set aside a code element or pulse at the 
end of a frame (consisting of a code word derived from each of the independent message 
sources in succession) and to transmit this pulse every other frame only. In such a case, 
the receiver includes a circuit that would search for the pattern of Is and Os alternating at 
half the frame rate, and thereby establish synchronization between the transmitter and 
receiver. 

When the transmission path is interrupted, it is highly unlikely that transmitter and 
receiver clocks will continue to indicate the same time for long. Accordingly, in carrying 
out a synchronization process, we must set up an orderly procedure for detecting the 
synchronizing pulse. The procedure consists of observing the code elements one by one 
until the synchronizing pulse is detected. That is, after observing a particular code element 
long enough to establish the absence of the synchronizing pulse, the receiver clock is set 
back by one code element and the next code element is observed. This searching process 
is repeated until the synchronizing pulse is detected. Clearly, the time required for syn- 
chronization depends on the epoch at which proper transmission is re-established. 



► Example 3.2 The Tl System 

In this example, we describe the important characteristics of a PCM system known as the Tl 
system, 6 which carries 24 voice channels over separate pairs of wires with regenerative re- 
peaters spaced at approximately 2-km intervals. The Tl carrier system is basic to the North 
American Digital Switching Hierarchy described in Section 3.10. 

A voice signal (male or female) is essentially limited to a band from 300 to 3100 Hz id 
that frequencies outside this band do not contribute much to articulation efficiency. Indeed 
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frequency of about 3.1 kHz prior to sampling. Hence, with W = Tl kHz the nn 1 > 

axis are spaced uniformly. Table ^sT ^S^^Z°^^£ 
horizonta axis and the step-sizes of the individual segments. The table is norma^ed to 159 
so that all values are represented as integer numbers. Segment 0 of the annrnvi™ J ' 

-3, . . . , 31, and the correspondmg representation levels at the quantizer output are 0 
-1, . . ±15. For hnear segments la and lb, the decision levels at the quantize/input are 

-16 ^17 " " " +3i m " eSP ° a,h l« mentation levels at the quantizer output are 

-«>, -i ±J1, and so on for the other hnear segments 

15 ™? " e " 'T 1 °1 31 + (W X U) = 255 ^Presentation levels associated with the 
15-segment companding characteristic described above. To accommodate thk n„™LT ( 

first bit indicates whether the input voice sample is positive or negative; this bit is a 1 if positive 

whlb tK Cg T/ e f T tthree ^ ° f theC ° de identify the particular se^ent 3 
wh.ch the amplitude of the input voice sample lies, and the last four bits identify^ Actual 
representation level inside that segment. laentiry tne actual 

With a sampling rate of 8 kHz, each frame of the multiplexed signal occupies a period 
of 125 ^ In particular ,t consists of twenty-four 8-bit words! plus a single bit 5ia is added 

total or (24 X 8) + 1 - 193 bits. Correspondingly, the duration of each bit equals 0 647 zls 
and the resulang transmission rate is 1.544 megabits per second (Mb/s) ^ 

signals^ttrend^Th^" T^ */^™ " mU " * h ° P aSS ^ -P-isory 
signals to the far end. This stgnahng information is needed to transmit dial pulses, as well as 

j-^^£li-j^!i!: $e ^ ewi companding characteristic O =255) 
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telephone off-hook/on-hook signals. In the Tl system, this requirement is accomplished as 
foUows Every sixth frame, the least significant (that is, the eighth) bit of each voice channel 
M^lZZnalingHt is inserted in its place, thereby yielding an average yf-bit operate 
or ach voice input. The sequence of signing bits is thus transmitted at a ra e equal to 
sampling rate of 8 kHz divided by six, that IS , 1.333 kb/s. This signaling rate applies to each 
of the 24 input channels. 

| 3.10 Digital Multiplexers 

In Section 3 9 we introduced the idea of time-division multiplexing whereby a group of 
analog "signals (e.g., voice signals) are sampled sequentially in tl me at a common sampling 
ratea^ 

le multiplexing of digital signals at different bit rates. This enables us to combine several 
El" Soak, such as computer outputs, digitized voice stgna s digitized facsimile, 
S t lelion signals, into a single data stream (at a considerably higher bit rate than 
Zy of the inputs). Figure 3.20 shows a conceptual diagram of the dtgrtal m uh,ple xmg - 

dem "TS^o]^ signals is accomplished by using a h^mterlea^ 
procedure with a selector switch that sequentially takes a bit from each incoming hne and 
fhTappUel to the high-speed common line. At the receiving end of the system the output 
of thk 'common line is separated out into its low-speed individual components and then 
delivered to their respective destinations. 

Digital multiplexers are categorized into two major groups. One group o multiplex- 
ers is used to take relatively low bit-rate data streams originating from digital computers 
and mu tiplex them for TDM transmission over the public switched telephone net- 
^ implementation of this first group of multiplexers requires the use of modems 
imnHnlators-demodulators), which are discussed in Chapter 6. 

' The second group of digital multiplexers forms part of the data transmission service 
provided by XommunicatL carriers such as AT6cT. In particular, these multiplexer, 
S I digital hierarchy that time-division multiplexes low-rate bit streams into mu h 
hS rate bit streams. The details of the bit rates that are accommodated m the hierarchy 
vaS from one country to another. However, a worldwide feature of the hierarchy is that 
[t IrtHt 64 kb/s, which corresponds to the standard PCM representation of a vo.c 
s^An uKommg bit stream at this rate, irrespective of its origin, is catted a d,„Ulsv£ 
To{DS0). In the United States, Canada, and Japan" the hierarchy follows the North 
American digital TDM hierarchy as described here: 

k The first-level hierarchy combines twenty-four DSO bit streams to obtain a digital 
signal L (DS1) at 1.544 Mb/s, which is carried on the Tl system described in 









High-speed 1 










2 




Multiplexer 




transmission 1 






line 



Demultiplexer 




Data sources 

FIGURE 3.20 Conceptual diagram of multiplexing-demultiplexing. 
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Example 3.2. These bit streams are called the primary rate in the digital hierarchy, 
because it is the lowest bit rate that exists outside a digital switch. The digital switch 
is a device consisting of memory and logic, the function of which is merely the switch- 
ing of digital signals, hence the name. 

>• The second-level multiplexer combines four DS1 bit streams to obtain a digital signal 
two (DS2) at 6.312 Mb/s. 

> The third-level multiplexer combines seven DS2 bit streams to obtain a digital signal 
three (DS3) at 44.736 Mb/s. 

*• The fourth-level multiplexer combines six DS3 bit streams to obtain a digital signal 
four (DS4) at 274.176 Mb/s. 

^ The fifth-level multiplexer, the final one in the hierarchy, combines two DS4 bit 
streams to obtain a digital signal five (DS5) at 560.160 Mb/s. 

Note that the bit rate of a digital signal produced by any one of these multiplexers is 
slightly higher than the prescribed multiple of the incoming bit rate because of bit stuffing 
built into the design of each multiplexer; bit stuffing is discussed in the sequel. 

Moreover, it is important to recognize that the functions of a digital transmission 
facility is merely to carry a bit stream without interpreting what the bits themselves mean. 
However, the digital switches at the two ends of the facility do have a common under- 
standing of how to interpret the bits within the stream, such as whether the bits represent 
voice or data, framing format, signaling format, and so on. 

There are some basic problems involved in the design of a digital multiplexer, irre- 
spective of its grouping: 

1. Digital signals cannot be directly interleaved into a format that allows for their even- 
tual separation unless their bit rates are locked to a common clock. Rather, provision 
has to be made for synchronization of the incoming digital signals, so that they can 
be properly interleaved. 

2. The multiplexed signal must include some form of framing so that its individual 
components can be identified at the receiver. 

3. The multiplexer has to handle small variations in the bit rates of the incoming digital 
signals. For example, a 1000-km coaxial cable carrying 3 X 10 8 pulses per second 
will have about one million pulses in transit, with each pulse occupying about one 
meter of the cable. A 0.01 percent variation in the propagation delay, produced by 
a 1°F decrease in temperature, will result in 100 fewer pulses in the cable. Clearly, 
these pulses must be absorbed by the multiplexer. 

To tailor the requirements of synchronization and rate adjustment to accommodate small 
variations in the input data rates, we may use a technique known as bit stuffing. The idea 
here is to have the outgoing bit rate of the multiplexer slightly higher than the sum of the 
maximum expected bit rates of the input channels by stuffing in additional non-informa- 
tion carrying pulses. All incoming digital signals are stuffed with a number of bits sufficient 
to raise each of their bit rates to equal that of a locally generated clock. To accomplish bit 
stuffing, each incoming digital signal or bit stream is fed into an elastic store at the mul- 
tiplexer. The elastic store is a device that stores a bit stream in such a manner that the 
stream may be read out at a rate different from the rate at which it is read in. At the 
demultiplexer, the stuffed bits must obviously be removed from the multiplexed signal. 
This requires a method that can be used to identify the stuffed bits. To illustrate one such 
method, and also show one method of providing frame synchronization, we describe the 
signal format of the AT&T Ml 2 multiplexer, which is designed to combine four DS1 bit 



216 Chapter 3 n Pulse Modulation 

streams into one DS2 bit stream. This is the second level of the digital hierarchy discu ssed 
earlier, 

► Example 3.3 Signal Format of the AT&T M12 Multiplexer 

Figure 3 21 illustrates the signal format of the M12 multiplexer Each frame is subdivided 
f„rrou; subframes. The fuS subframe (first line in Figure 3.21) is transacted, then the 
sprond the third, and the fourth, m that order. 

B*by bit interleaving of the incoming four DS1 bit streams is used to accumulate a 
< , 1 f 48 hits 12 from each input. A control bit is then inserted by the multiplexer. Each 
SIS.Sm control bits, separated by sequences of 48 data bit. Three types 
ofTonttol Z are used in the M12 multiplexer to provide synchronization and frame md , 
ftLn and to identify which of the four input signals has been stuffed. These control bits are 
labeled F, M, and C in Figure 3.21. Their functions are as follows: 

1 The F-control bits, two per subframe, constitute the main framing pulses. The subscripts 
In the Control bit, denote the actual bit (0 or 1) transmitted. Thus the main framing 
sequence is WWM or 01010101. 

2 The Al-control bits, one per subframe, form secondary framing pulses to identify At 
2 - Itlframes. Here agl the subscripts on the M-control ^jeno« the actual bit 

(0 or 1) transmitted. Thus the secondary framing sequence is AMMMJ, ot 0111 
1 The C-control bits, three per subframe, are stuffing indicators. In particular, C, refers 
I npr^nnc I C U refers to input channel II, and so forrh. For example, the three 
cZtto bhs in the first subframe following M„ in the first subframe are sniffing in- 
dSo"s lor tl first DS1 bit stream. The insertion of a stuffed bit m this DS1 bit stream 
fs tad° ated D y setting all three C-control bits to 1. To indicate no stuffing, all three ar 
e to 0 If the three C-control bits indicate stuffing, the stuffed bit is located in the 
postln o tt fi7st information bit associated with the first DS1 bit stream .that ^ows 
the ^ control bit in the same subframe. In a similar way, the second third and four* 
DS1 bit Yearns m ay be stuffed, as required. By using majority ogre decoding m the 
fe iv atngle error in any of the three C-control bits can be detected. Tto form of 
decXg means simply that the majority of the C-control bits determine wh ther an 
all-on or all-zero sequence was transmitted. Thus three Is or combination s oU »o U 
2d a 0 indicate that a stuffed bit is present in the information sequence, lowing A 
control Wt Fi in the pertinent subframe. On the other hand, three 0s or combinations 
of two Os and a 1 indicate that no stuffing is used. 
The demultiplexer at the receiving M12 unit first ^^^ffi' 
WiFoWiVi. This " tablishes identk y for ^ f0UI T bit yearns and I also lor 
S and C-control bits. From the M oMl M, Ml sequence, the correct ^framing °f *e « 
bits is verified. Finally, the four DS1 bit streams are properly demultiplexed and destuffed. 
The signal format described above has two safeguards: 

1. It is possible, although unlikely, that with just rhe WWW 

the incoming DS1 bit streams may contain a similar sequence. This could then 
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Figure 3.21 Signal format of AT&T Ml 2 multiplexer. 
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the receiver to lock onto the wrong sequence. The presence of the M 0 M,MiMi sequence 
provides verification of the genuine FoF^aF^gF^Fi sequence, thereby ensuring that 
the four DS1 bit streams are properly demultiplexed. 
2. The single-error correction capability built into the C-control bits ensures that the four 
DS1 bit streams are properly destuffed. 

The capacity of the M12 multiplexer to accommodate small variations in the input data 
rates can be calculated from the format of Figure 3.21 . In each M frame, defined as the interval 
containing one cycle of MqM^MJAi bits, one bit can be stuffed into each of four input DS1 
bit streams. Each such signal has 12 X 6 X 4 = 288 positions in each M frame. Also, the Tl 
bit stream has a bit rate equal to 1.544 Mb/s. Hence, each input can be incremented by 

1.S44 X 10 3 X -i- = 5.4 kb/s 
288 

This result is much larger than the expected change in the bit rate of the incoming DS1 bit 
stream. It follows therefore that the use of only one stuffed bit per input channel in each frame 
is sufficient to accommodate expected variations in the input signal rate. 

The local clock that determines the outgoing bit rate also determines the nominal stuffing 
rate S, defined as the average number of bits stuffed per channel in any frame. The 
M12 multiplexer is designed for S = 1/3. Accordingly, the nominal bit rate of the DS2 bit 
stream is 

49 288 
1 - 544 X 4 X 48 X 28^S = 6 - 312Mb/S 

This also ensures that the nominal DS2 clock frequency is a multiple of 8 kHz (the nominal 
sampling rate of a voice signal), which is a desirable feature. 



13.1 1 Virtues, Limitations, 
and Modifications of PCM 

In a generic sense, pulse-code modulation (PCM) has emerged as the most favored mod- 
ulation scheme for the transmission of analog information-bearing signals such as voice 
and video signals. The advantages of PCM may all be traced to the use of coded pulses 
for the digital representation of analog signals, a feature that distinguishes it from all other 
analog methods of modulation. We may summarize the important advantages of PCM as 
follows: 

1. Robustness to channel noise and interference. 

2. Efficient regeneration of the coded signal along the transmission path. 

3. Efficient exchange of increased channel bandwidth for improved signal-to-noise ra- 
tio, obeying an exponential law. 

4. A uniform format for the transmission of different kinds of baseband signals, hence 
their integration with other forms of digital data in a common network. 

5. Comparative ease with which message sources may be dropped or reinserted in a 
time-division multiplex system. 

6. Secure communication through the use of special modulation schemes or encryption; 
the encryption and decryption of data are discussed in Appendix 5. 

These advantages, however, are attained at the cost of increased system complexity and 
increased channel bandwidth. These two issues are considered in the sequel in turn. 
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Although the use of PCM involves many complex operations, today they can all b e 
implemented in a cost-effective fashion using commercially available and/or custom-made 
very-large-scale integrated (VLSI) chips. In other words, the requisite device technology 
for the implementation of a PCM system is already in place. Moreover, with continuing 
improvements in VLSI technology, we are likely to see an ever-expanding use of PCM f ot 
the digital transmission of analog signals. 

If, however, the simplicity of implementation is a necessary requirement, then We 
may use delta modulation as an alternative to pulse-code modulation. In delta modulation, 
the baseband signal is intentionally "oversampled" to permit the use of a simple quantizing 
strategy for constructing the encoded signal; delta modulation is discussed in Section 3.12, 

Turning next to the issue of bandwidth, we do recognize that the increased band- 
width requirement of PCM may have been a reason for justifiable concern in the past. 
Today, however, it is of no real concern for two different reasons. First, the increasing 
availability of wideband communication channels means that bandwidth is no longer a 
system constraint in the traditional way it used to be. Liberation from the bandwidth 
constraint has been made possible by the deployment of communication satellites for 
broadcasting and the ever-increasing use of fiber optics for networking; a discussion of 
these communication channel concepts was presented in the Background and Preview 

chapter. . 

The second reason is that through the use of sophisticated data compression tech- 
niques, it is indeed possible to remove the redundancy inherently present in a PCM signal 
and thereby reduce the bit rate of the transmitted data without serious degradation in 
system performance. In effect, increased processing complexity (and therefore increased 
cost of implementation) is traded off for a reduced bit rate and therefore reduced band- 
width requirement. A major motivation for bit-rate reduction is for secure communication 
over radio channels that are inherently of low capacity. 



3.12 Delta Modulation 

In delta modulation 3 (DM), an incoming message signal is oversampled (i.e., at a rate 
much higher than the Nyquist rate) to purposely increase the correlation between adjacent 
samples of the signal. This is done to permit the use of a simple quantizing strategy for 
constructing the encoded signal. 

In its basic form, DM provides a staircase approximation to the oversampled version 
of the message signal, as illustrated in Figure 3.22a. The difference between the input and 
the approximation is quantized into only two levels, namely, ± A, corresponding to positive 
and negative differences. Thus if the approximation falls below the signal at any sampling 
epoch, it is increased by A. If on the other hand, the approximation lies above the signal, 
it is diminished by A. Provided that the signal does not change too rapidly from sample 
to sample, we find that the staircase approximation remains within ± A of the input signal. 

Let m(t) denote the input (message) signal, and m q {t) denote its staircase approxi- 
mation. For convenience of presentation, we adopt the following notation that is com- 
monly used in the digital signal processing literature: 

m[n]=m(nT s ), n = 0, ±1, ±2, . . . 

where T„ is the sampling period and m(nT s ) is a sample of the signal m(t) taken at time 
£ = nT s , and likewise for the samples of other continuous-time signals. We may then 
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Figure 3.22 Illustration of delta modulation. 



formalize the basic principles of delta modulation in the following set of discrete-time 
relations: 



where e[n] is an error signal representing the difference between the present sample m[n] 
of the input signal and the latest approximation m q [n — 1] to it, ejn] is the quantized 
version of e[n], and sgn(-) is the signum function. Finally, the quantizer output m q [n] is 
coded to produce the DM signal. 

Figure 3.22fl illustrates the way in which the staircase approximation m q (t) follows 
variations in the input signal m(t) in accordance with Equations (3.52)-(3.54), and Figure 
3.22b displays the corresponding binary sequence at the delta modulator output. It is 
apparent that in a delta modulation system the rate of information transmission is simply 
equal to the sampling rate f s = 1/T S . 

The principal virtue of delta modulation is its simplicity. It may be generated by 
applying the sampled version of the incoming message signal to a modulator that involves 
a comparator, quantizer, and accumulator interconnected as shown in Figure 3.23fl. The 
block labeled z~* inside the accumulator represents a unit delay, that is, a delay equal to 
one sampling period. (The variable z is commonly used in the z-transform, which is basic 
to the analysis of discrete-time signals and systems.) Details of the modulator follow di- 
rectly from Equations (3.52)-(3.54). The comparator computes the difference between its 
two inputs. The quantizer consists of a hard limiter with an input-output relation that is 
a scaled version of the signum function. The quantizer output is then applied to an accu- 
mulator, producing the result 




m[n] — m q [n — J] 

A sgn(e[»]) 

»,[» - 1] + e q [n] 



(3.52) 
(3.53) 
(3.54) 



m q [n] 



A 2! sgn(eH) 



i=1 



(3.55) 
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which is obtained by solving Equations (3.53) and (3.54) for «,[«]. Thus, at the sampling 
ant «T the accumulator increments the approximation by a step A in a positive « 
neSe ii ection, depending on the algebraic sign of the error sample e[n\ If the inpu 
"mple «W is greater than the most recent approximation * M , a positive incremen 
+Ms applied to the approximation. If, on the other hand, the mput sample is small r, a 
neg^ivetrrement Ji applied to the approximation. In this way, Je accumulator does 
the best it can to track the input samples by one step (of amplitude + A or A) at a time. 
In the receiver shown in Figure 3.23b, the staircase approximation m q{ t ) is reconsttuc. d 
by passing the sequence of positive and negative pulses, produced at the decoder outpu , 
an accumulator in a manner similar to that used in the transmitter The ou -of- 
band quantization noise in the high-frequency sta.rcase waveform «,(*). s re,e ted * 
passing it through a low-pass filter, as in Figure 3.23b, with a bandwidth equal to the 

^^oristbiec, to ,o types of quantization error: slope — * 
tortion and granular noise. We will discuss the case of slope overload distortion farst 

We obferve that Equation (3.54) is the digital equivalent of integration in the ser,e 
that it represents the accumulation of positive and negative increments of magnitude A. 
Also, denoting the quantization error by q[n], as shown by 



m q [n\ = m[n] + q[n] ' 3-56 ' 



we observe from Equation (3.52) that the input to the quantizer is 



e[n] = m[n] - m[n - 1] - q[n - 1] (3 ' 571 
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FIGURE 3.24 Illustration of the two different forms of quantization error in delta modulation. 



Thus except for the quantization error q[n - 1], the quantizer input is a first backward 
difference of the input signal, which may be viewed as a digital approximation to the 
derivative of the input signal or, equivalently, as the inverse of the digital integration 
process. If we consider the maximum slope of the original input waveform m(t), it is 
clear that in order for the sequence of samples {m q [n]\ to increase as fast as the input 
sequence of samples {m[n]} in a region of maximum slope of m(t), we require that the 
condition 

A 



be satisfied. Otherwise, we find that the step-size A is too small for the staircase approxi- 
mation m q (t) to follow a steep segment of the input waveform m(t), with the result that 
m q (t) falls behind m(t), as illustrated in Figure 3.24. This condition is called slope overload, 
and the resulting quantization error is called slope-overload distortion (noise). Note that 
since the maximum slope of the staircase approximation m q (t) is fixed by the step size A, 
increases and decreases in m q (t) tend to occur along straight lines. For this reason, a delta 
modulator using a fixed step size is often referred to as a linear delta modulator. 

In contrast to slope-overload distortion, granular noise occurs when the step size A 
is too large relative to the local slope characteristics of the input waveform m(t), thereby 
causing the staircase approximation tn q (t) to hunt around a relatively flat segment of the 
input waveform; this phenomenon is also illustrated in Figure 3.24. Granular noise is 
analogous to quantization noise in a PCM system. 

We thus see that there is a need to have a large step-size to accommodate a wide 
dynamic range, whereas a small step size is required for the accurate representation of 
relatively low-level signals. It is therefore clear that the choice of the optimum step size 
that minimizes the mean-square value of the quantization error in a linear delta modulator 
will be the result of a compromise between slope-overload distortion and granular noise. 
To satisfy such a requirement, we need to make the delta modulator "adaptive," in the 
sense that the step size is made to vary in accordance with the input signal; this issue is 
discussed further in a computer experiment presented in Section 3.16. 

■ Delta-Sigma Modulation 

As mentioned earlier, the quantizer input in the conventional form of delta modulation 
may be viewed as an approximation to the derivative of the incoming message signal. This 
behavior leads to a drawback of delta modulation in that transmission disturbances such 
as noise result in an accumulative error in the demodulated signal. This drawback can be 



dm(t) 



dt 



(3.58) 
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overcome by integrating the message signal prior to delta modulation The use of Integra- 
tion in the manner described here has also the following beneficial effects. 

► The low-frequency content of the input signal is pre-emphasized. 

► Correlation between adjacent samples of the delta modulator input is mcreased, 
which tends to .mprove overall system performance by reducing the variance of the 
error signal at the quantizer input. 

► Design of the receiver is simplified. 

A delta modulation scheme that incorporates integration ^ ^^^^"1 
Modulation (D-2M). 9 To be more precise, however, it should be called stgma-dehamod- 
JaZZZLse the integration is in fact performed before the delta modulation. Neve:- 
the ess the former terminology is the one commonly used in the literature. 

Figu e 3 25a shows the block diagram of a delta-sigma modulation system. In thus 
diagram the message signal m(f ) is defined in its continuous-time form, which means that 
1 modulator now consists of a hard-limiter followed by a multiplier; the latter 
components also fed from an external pulse generator (dock) to produce a 1-bit encoded 
Zl Th use of integration at the transmitter input clearly requires an inverse signal 
eZhas s namely, differentiation, at the receiver. The need for this differentiation ,s, how- 
e^eStTdb cause of its cancellation by integration in the conventional DM recover. 
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Figure 3.25 Two equivalent versions of delta-sigma modulation system. 
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Thus the receiver of a delta-sigma modulation system consists simply of a low-pass filter, 
as indicated in Figure 3.25a. 

Moreover, we note that integration is basically a linear operation. Accordingly, we 
may simplify the design of the transmitter by combining the two integrators 1 and 2 of 
Figure 3.25a into a single integrator placed after the comparator, as shown in Figure 3.25i>. 
This latter form of the delta-sigma modulation system is not only simpler than that of 
Figure 3.25a, but it also provides an interesting interpretation of delta-sigma modulation 
as a "smoothed" version of 1-bit pulse-code modulation: The term smoothness refers to 
the fact that the comparator output is integrated prior to quantization, and the term 1-bit 
merely restates that the quantizer consists of a hard-limiter with only two representation 
levels. 

In delta modulation, simplicity of implementations of both the transmitter and re- 
ceiver is attained by using a sampling rate far in excess of that needed for pulse-code 
modulation. The price paid for this benefit is a corresponding increase in the transmission 
and therefore channel bandwidth. There are, however, applications where channel band- 
width is at a premium, in which case we have the opposite requirement to that in delta 
modulation. Specifically, we may wish to trade increased system complexity for a reduced 
channel bandwidth. A signal-processing operation basic to the attainment of this latter 
design objective is prediction, the linear form of which is discussed next. 



I 



3.13 Linear Prediction 



Consider a finite-duration impulse response (FIR) discrete-time filter configured as in Fig- 
ure 3.26, which involves the use of three functional blocks: 

1. Set of p unit-delay elements, each of which is represented by z^ 1 . 

2. Set of multipliers involving the filter coefficients w x , w 2 , . . . , w p . 

3. Set of "adders" used to sum the scaled versions of the delayed inputs x[n — 1], 
x[n — 2], ... , x[n - p\ to produce the output x[n]. The filter output x[n] or more 
precisely, the linear prediction of the input, is thus defined by the convolution sum 

p 

x[n] = 2 w k x]n - k] (3.59) 

k-i 

where p, the number of unit-delay elements, is called the prediction order. 

The actual sample at time nT s is x[n]. The prediction error, denoted by e[n], is defined 
as the difference between x[n] and the prediction x[n], as shown by 



e[n] = x[n] - x[n] 



(3.60) 
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Figure 3.26 Block diagram of a linear prediction filter of order p. 
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The design objective is to choose the filter coefficients w lt w z ,...,w p so as to minimize 
an index of performance, ] , defined as the mean-square error: 



(3.61) 



Substituting Equations (3.59) and (3.60) into (3.61) and then expanding terms, we may 
reformulate the index of performance as 

P P p 

J = E\x 2 \n]] - 2 E w k E[x[n)x[n - fe]] + 2 2 W/U/ k E[x[n - /M" " *]] (3.62) 

We assume that the input signal x(t) is the sample function of a stationary process 
X(t) of zero mean; that is, E[x[n]] is zero for all n. Define 

oi = variance of a sample of the process X{t) at time bT, 
= £[*>]] - (E[*[»]]) 2 
= E[* 2 [«]] 

Rx(^T 5 ) = autocorrelation of the process X(t) for a lag of kT s 
= Rx[fe] 

= E[x[n]x[n - k]] 
Accordingly, we may rewrite Equation (3.62) in the simplified form 

p p p 

J = a 2 x ~ 2 2 WkRxik] + S2 w jWk K x [k- /] 



(3.63) 



Hence differentiating the index of performance / with respect to the filter coefficient w h 
setting the result equal to zero, and then rearranging terms, we obtam 



2 w,Rxl k - fl = R x [k] = Rxl-k], k = l,2,...,p 



(3.64) 



The optimality equations (3.64) are called the Wiener-Hop f equations for linear prediction. 

We find it convenient to reformulate the Wiener-Hopf equations (3.64) in matrix 
form. Let 

Wo = p-by-1 optimum coefficient vector 

= [if i, U>2, ■ ■ ■ , w p ] T 
tx = p-by-1 autocorrelation vector 

= [R x [l], R x [2], Rx\p]] T 
R x = p-by-p autocorrelation matrix 
R x [0] RxW ■ ■ ■ 
Rx[1] Rx[0] 



RxlP ~ 1] RxlP - 2 ] 
We may thus simplify the set of equations (3.64) as 

R x w„ = r x 



RxlP - }]' 
RxlP - 2] 

MO] . 



(3.65) 
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We assume that the autocorrelation matrix R x is nonsingular, so that its inverse exists 
We may then solve Equation (3.65) for the coefficient vector w 0 by multiplying both sides 
of this equation by the inverse matrix R* 1 , obtaining the optimum solution 

w G = R x 1 r x (3,66) 

Note that all the elements on the main diagonal of the autocorrelation matrix R x 
are equal to R x [0] = oi, and the elements on any other diagonal parallel to the main 
diagonal are also equal. A square matrix having this property is said to be Toeplitz, which 
is a direct consequence of the assumption that the input signal x(t) is drawn from a sta- 
tionary process. The practical significance of the Toeplitz property is that the correlation 
matrix R x is uniquely defined by the set of autocorrelation values R x [0], R x [l], ■ 
•Kx[p -}]■ The autocorrelation vector r x is denned by the set of autocorrelation values' 
Rx[l], R x [2], • • • , Rxlp]- It follows therefore that the p filter coefficients of the linear 
optimum predictor are uniquely defined by the variance oi = R x [0] and p values of the 
autocorrelation function of the process X(t) for lags of T„ 2T S , pT s . 

The minimum mean-square value of the prediction error is obtained by substituting 
Equation (3.64) into (3.63), which yields (after simplification) 

Jmm = ox - r x R x J r x (3.67) 
The quadratic term r x R x J r x is always positive. Accordingly, the mean-square error / min of 
the optimum linear predictor defined by Equation (3.67) is always less than the variance 
<r x of the input sample that is being predicted. 



■ Linear Adaptive Prediction 

The use of Equation (3.66) for calculating the weight vector of a linear predictor requires 
knowledge of the autocorrelation function R x [k] of the input sequence [x[n]} for lags 
k = 0, 1, . . . , p, where p is the prediction order. What if knowledge of R x [k] for varying 
k is not available? In these situations, which occur frequently in practice, we may resort 
to the use of an adaptive predictor. 

The predictor is adaptive in the following sense: 

► Computation of the tap weights w k , k = 1, 2, . . . , p, proceeds in a "recursive" 
manner, starting from some arbitrary initial values of the tap weights. 

* Tne algorithm used to adjust the tap weights (from one iteration to the next) is "self- 
designed," operating solely on the basis of available data. 

The aim of the algorithm is to find the minimum point of the bowl-shaped error surface 
that describes the dependence of the cost function / on the tap weights. It is therefore 
intuitively reasonable that successive adjustments to the tap-weights of the predictor be 
made in the direction of the steepest descent of the error surface, that is, in a direction 
opposite to the gradient vector whose elements are defined by 

g * = £? k = 1, 2, . . . , p (3.68) 

This is indeed the idea behind the method of steepest descent. Let w k [n] denote the value 
of the kth tap-weight at iteration n. Then the updated value of this weight at iteration 
n + 1 is defined by 



w k [n + 1] = w k [n] -jixg k , k = l,2,...,p 



(3.69) 
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where p. is a step-size parameter that controls the speed of adaptation, and the factor \\i 
is included for convenience of presentation. Differentiating the cost function/ of Equation 
(3.63) with respect to w k , we readily find that 

gk = -2R x [k] + 2 £ wfi x [k ~ i\ 

' _1 , (3.70) 

= -2E\x\n]x[n - k]] + 2 £ Wi E[x[n - j]x[n - k]], k = l,2,...,p 

/-i 

This formula for g k could do with further simplification, which is achieved by using 
stantaneous values as estimates of the autocorrelation functions R x [k] and R x [k - /]. That 
is, we ignore the expectation operators in Equation (3.70) to facilitate the adaptive process 
on a step-by-step basis. We may thus express the corresponding estimate oig k at iteration 



n as 



p 

g k [n] = -2x[n]x[n - k] + 2 ]£ w,[n\x[n - j]x[n - k], k = l,2,...,p (3.71) 
i-i 

Note that for an input x[n] drawn from a stationary process the gradient g k is a determin- 
istic quantity, whereas the estimate gk [n\ is the sample value of a random variable. 

In any event, substituting Equation (3.71) into (3.69) and factoring the common 
term x[n - k], we may write 

ii k [n + 1] = iv k [n\ + ixx[n - k\(x[ii\ - 2 w,[n\x[n - j\\ 

\ i-" 1 ' (3.72) 

= w k [n] + ixx[n - k]e[n], k = 1, 2, . . . , p 

where e[n] is the prediction error defined as 

p 

e\n\ = x[n] - S ^;[»M" - /] < 3 - 73 > 

/-i 

In Equations (3.72) and (3.73), we have used w k as an estimate of the kth tap-weightto 
distinguish it from the actual value w k . Note also that x[n] plays the role of a "desired 
response" for computing the recursive adjustments applied to the tap-weights of the 
predictor. , 

Equations (3.72) and (3.73) constitute the popular least-mean-square (LMSJ algo- 
rithm for linear adaptive prediction, the operation of which is depicted in Figure 3.27. The 
reason for popularity of this adaptive filtering algorithm is the simplicity of its implemen- 
tation. In particular, the computational complexity of the algorithm, measured in terms 
of the number of additions and multiplications, is linear in the prediction order p. 



Input 
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Linear predictor: 

{- t M} 



Figure 3.27 Block diagram illustrating the linear adaptive prediction process. 
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The LMS algorithm is a stochastic adaptive filtering algorithm, stochastic in the sense 
that, starting from the initial condition defined by {w k [0]&-i, it seeks to find the minimum 
point of the error surface by following a zig-zag path. Moreover, it never finds this mini- 
mum point exactly. Rather, it executes a random motion around the minimum point of 
the error surface, once steady-state conditions are established. 

With this material on linear prediction at hand, we are ready to discuss practical 
improvements on the performance of pulse-code modulation. 



L 



3.14 Differential Pulse-Code Modulation 



When a voice or video signal is sampled at a rate slightly higher than the Nyquist rate as 
usually done in pulse-code modulation, the resulting sampled signal is found to exhibit a 
high degree of correlation between adjacent samples. The meaning of this high correlation 
is that, in an average sense, the signal does not change rapidly from one sample to the 
next, and as a result, the difference between adjacent samples has a variance that is smaller 
than the variance of the signal itself. When these highly correlated samples are encoded, 
as in the standard PCM system, the resulting encoded signal contains redundant infor- 
mation. This means that symbols that are not absolutely essential to the transmission of 
information are generated as a result of the encoding process. By removing this redundancy 
before encoding, we obtain a more efficient coded signal, which is the basic idea behind 
differential pulse-code modulation. 

Now if we know the past behavior of a signal up to a certain point in time, we may 
use prediction to make an estimate of a future value of the signal as described in Section 
3.13. Suppose then a baseband signal m(t) is sampled at the rate f s = 1/T, to produce the 
sequence {>»[«]} whose samples are T s seconds apart. The fact that it is possible to predict 
future values of the signal m(t) provides motivation for the differential quantization scheme 
shown in Figure 3.28a. In this scheme, the input signal to the quantizer is defined by 

e[n\ = m[n] - m[n] (3.74) 

which is the difference between the unquantized input sample m[n] and a prediction of it, 
denoted by m[n}. This predicted value is produced by using a linear prediction filter whose 
input, as we will see, consists of a quantized version of the input sample m\n\. The differ- 
ence signal e[n] is the prediction error, since it is the amount by which the prediction filter 
fails to predict the input exactly. By encoding the quantizer output, as in Figure 3.28a, we 
obtain a variant of PCM known as differential pulse-code modulation 10 (DPCM). 
The quantizer output may be expressed as 

e q [n] = e [n] + q[n] (3.7j) 

where q[n] is the quantization error. According to Figure 3.28a, the quantizer output e q [n] 
is added to the predicted value m[n] to produce the prediction-filter input ? 

m q [n] = m\n] + e q [n] (3.76) 

Substituting Equation (3.75) into (3.76), we get 

m q [n] = m[n] + e[n] + q[n] (3,77) 

However, from Equation (3.74) we observe that the sum term m[n] + e\n] is equal to the 
input sample m[n]. Therefore, we may simplify Equation (3.77) as 

m q [n] = m[n] + q[n] (3.78) 
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FIGURE 3.28 DPCM system, (a) Transmitter, (b) Receiver. 



which represents a quantized version of the input sample «[»]. That is, irrespective of the 
propert es of the prediction filter, the quantized sample »,[»] at the prediction filter mpu 
d ffers from the original input sample «[»] by the quantization error Accor^M 
the prediction is good, the variance of the prediction error e[n] wil be smaller than the 
variance *[»] so that a quantizer with a given number of levels can be admstedto 
produce a quantization error with a smaller variance than would be possible if the input 
sample m\n] were quantized directly as in a standard PCM system. 

The receiver for reconstructing the quantized version of the mput .s shown m Fig 
3 28b It consists of a decoder to reconstruct the quantized error signal. The quantize 
version of the original input is reconstructed from the decoder output using th same 
^diction filter used in the transmitter of Figure 3.28*. In the absence of channel mm, 
we find that the encoded signal at the receiver mput is identical to the encoded Signal* 
Te ttansmitter output. Accordingly, the corresponding receiver output is equal to m 4 
whi^ffe torn the original mput «[»] only by the quantization error q [n] incurred* 
a result of quantizing the prediction error e[n]. 

From the foregoing analysis we observe that, in a no.se-free environmen the pr, 
diction filters in the transmitter and receiver operate on the same sequence o samp es 
n^n] It is with this purpose in mind that a feedback path is added to the quantizer inthe 
transmitter, as shown in Figure 3.28fl. . . i 

Differential pulse-code modulation includes delta modulation , as a special c^b 
particular, comparing the DPCM system of Figure 3.28 with the DM system of Figure 
3 23, we see that they are basically similar, except for two important 
of a one-bit (two-level) quantizer in the delta modulator and the -pkce-n of th £ 
diction filter by a single delay element (i.e., zero prediction order). Simply put^ DM i 
Tbit version of DPCM. Note that unlike a standard PCM system, the transmitters of bo* 
the DPCM and DM involve the use of feedback. ^ 

DPCM, like DM, is subject to slope-overload distortion whenever the input sig 
changes too rapidly for the prediction filter to track it. Also, like PCM, DPCM 
from quantization noise. 
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m Processing Gain 

The output signal-to-noise ratio of the DPCM system shown in Figure 3.28 is, by 
definition, 

(SNR)o = ^ (3.79) 

where oj^ is the variance of the original input sample m[n], assumed to be of zero mean, 
and oq is the variance of the quantization error q[n\. We may rewrite Equation (3.79) as 
the product of two factors as follows: 



(SNR)o 



0 i/\°Q/ (3.80) 



G„(SNR) Q 



where aj, is the variance of the prediction error. The factor (SNR) Q is the signal-to- 
quantization noise ratio, which is defined by 

(SNR)q = (3.81) 

The other factor G p is the processing gain produced by the differential quantization 
scheme; it is defined by 



4 



G P = ^t (3-82) 



The quantity G p , when greater than unity, represents a gain in signal-to-noise ratio that 
is due to the differential quantization scheme of Figure 3.28. Now, for a given baseband 
(message) signal, the variance oj^ is fixed, so that G p is maximized by minimizing the 
variance <r| of the prediction error e[n]. Accordingly, our objective should be to design 
the prediction filter so as to minimize a\. 

In the case of voice signals, it is found that the optimum signal-to-quantization noise 
advantage of DPCM over standard PCM is in the neighborhood of 4 to 1 1 dB. The greatest 
improvement occurs in going from no prediction to first-order prediction, with some ad- 
ditional gain resulting from increasing the order of the prediction filter up to 4 or 5, after 
which little additional gain is obtained. Since 6 dB of quantization noise is equivalent to 
1 bit per sample by virtue of Equation (3.35), the advantage of DPCM may also be ex- 
pressed in terms of bit rate. For a constant signal-to-quantization noise ratio, and assuming 
a sampling rate of 8 kHz, the use of DPCM may provide a saving of about 8 to 16 kb/s 
(i.e., 1 to 2 bits per sample) compared to the standard PCM. 



3.15 Adaptive Differential 
Pulse-Code Modulation 

The use of PCM for speech coding at the standard rate of 64 kb/s demands a high channel 
bandwidth for its transmission. In certain applications, however, such as secure transmis- 
sion over radio channels that are inherently of low capacity, channel bandwidth is at a 
premium. In applications of this kind, there is a definite need for speech coding at low bit 
rates, while maintaining acceptable fidelity or quality of reproduction. 
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For coding speech at low bit rates, a waveform coder of prescribed configuration j, 
optimized by exploiting both statistical characterization of speech waveforms and prop, 
erties of hearing. In particular, the design philosophy has two aims in mind: 

1. To remove redundancies from the speech signal as far as possible. 

2. To assign the available bits to code the nonredundant parts of the speech signal in a 
perceptually efficient manner. 

As we strive to reduce the bit rate from 64 kb/s (used in standard PCM) to 32, 16, 8, and 
4 kb/s, the schemes used for redundancy removal and bit assignment become increasingly 
more sophisticated. As a rule of thumb, in the 64 to 8 kb/s range, the computational 
complexity (measured in terms of multiply-add operations) required to code speech in- 
creases by an order of magnitude when the bit rate is halved, for approximately equal 
speech quality. 

In this section, we describe adaptive differential pulse-code modulation (ADPCM), 11 
which permits the coding of speech at 32 kb/s through the combined use of adaptive 
quantization and adaptive prediction; the number of eight bits per sample required in the 
standard PCM is thereby reduced to four. The term adaptive used herein means being 
responsive to changing level and spectrum of the input speech signal. The variation of 
performance with speakers and speech material, together with variations in signal level 
inherent in the speech communication process, make the combined use of adaptive quan- 
tization and adaptive prediction necessary to achieve best performance over a wide range 
of speakers and speaking situations. 

Adaptive quantization refers to a quantizer that operates with a time-varying step- 
size A[«]. At any given sampling instant identified by the index n, the adaptive quantizer 
is assumed to have a uniform transfer characteristic. The step-size A[n] is varied so as to 
match the variance oil of the input sample m[n]. In particular, we write 

A[n] = <f,& M [n] (3.83) 

where <j> is a constant, and a M [n] is an estimate of the standard deviation cr M [«] (i.e., square 
root of the variance of,). For a nonstationary input, cr M [n] is time varying. The problem 
of adaptive quantization according to Equation (3.83) is, therefore, one of computing tie 
estimate omM continuously. 

The implementation of Equation (3.83) may proceed in one of two ways: 

1. Adaptive quantization with forward estimation (AQF), in which unquantized sam- 
ples of the input signal are used to derive forward estimates of cr M [n]. 

2. Adaptive quantization with backward estimation (AQB), in which samples of tie 
quantizer output are used to derive backward estimates of a- M [n], 

The AQF scheme requires the use of a buffer to store unquantized samples of the input 
speech signal needed for the learning period. It also requires the explicit transmission of 
level information (typically, about 5 to 6 bits per step-size sample) to a remote decoder, 
thereby burdening the system with additional side information that has to be transmitted 
to the receiver. Moreover, a processing delay (on the order of 16 ms for speech) in the 
encoding operation results from the use of AQF, which is unacceptable in some appfe' 
tions. The problems of level transmission, buffering, and delay intrinsic to AQF are all 
avoided in AQB. In the latter scheme, the recent history of the quantizer output is used to 
extract information for the computation of the step size A[n]. In practice, AQB is therefore 
usually preferred over AQF. 

Figure 3.29 shows the block diagram of an adaptive quantizer with backward esti- 
mation. It represents a nonlinear feedback system; hence, it is not obvious that the system 
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Figure 3.29 Adaptive quantization with backward estimation (AQB). 



will be stable. However, if the quantizer input m[n] is bounded, then the backward estimate 
CmM and the corresponding step size A[»] are as well; under such a condition, the system 
is indeed stable. 

The use of adaptive prediction in ADPCM is justified because speech signals are 
inherently nonstationary, a phenomenon that manifests itself in the fact that the autocor- 
relation function and power spectral density of speech signals are time-varying functions 
of their respective arguments. This implies that the design of predictors for such inputs 
should likewise be time varying, that is, adaptive. As with adaptive quantization, there are 
two schemes for performing adaptive prediction: 

1. Adaptive prediction with forward estimation (APF), in which unquantized samples 
of the input signal are used to derive estimates of the predictor coefficients. 

2. Adaptive prediction with backward estimation (APB), in which samples of the quan- 
tizer output and the prediction error are used to derive estimates of the predictor 
coefficients. 

However, APF suffers from the same intrinsic disadvantages (side information, buffering, 
and delay) as AQF. These disadvantages are eliminated by using the APB scheme shown 
in Figure 3.30, where the box labeled "logic for adaptive prediction" represents the al- 
gorithm for updating the predictor coefficients. In the latter scheme, the optimum predictor 
coefficients are estimated on the basis of quantized and transmitted data; they can therefore 
be updated as frequently as desired, say, from sample to sample. Accordingly, APB is the 
preferred method of prediction for ADPCM. 

The LMS algorithm for the predictor, described in Section 3.13, and an adaptive 
scheme for the quantizer, based on Equation (3.83), have been combined in a synchronous 
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Figure 3.30 Adaptive prediction with backward estimation (APB). 
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fashion for the design of both the encoder and decoder. The performance of this combi- 
nation is so impressive at 32 kb/s that ADPCM is now accepted -t^o^as a sta , 
dard coding technique for voice signals, along with 64 kb/s using standard PCM. 



3.16 Computer Experiment: 
Adaptive Delta Modulation 

A simple form of AQB is to be found in the modification of linear delta modulation (LDM) 
i^maZptive delta modulation (ADM). The principle underlying all ADM algorithms 
is two-fold: 

1 If successive errors are of opposite polarity, then the delta modulator is operating i„ 
its granular mode; in this case, it may be advantageous to reduce the step size. 

2 If however, successive errors are of the same polarity, then the delta modulator „ 
operating in its slope-overload mode; in this second case, the step sue should be 
increased. 

Thus by varying the step-size in accordance with this principle, the delta modulator is 
enabled to cope with changes in the input signal. 

Fig^e 3 31 shows the block diagram of an ADM based on increasing or decreasing 
the steppe by a factor of 50 percent at each iteration of the adaptive process. The d- 
gorithm for adaptation of the step size is defined by 

ifciil ( m [B] + 0 .5«,[» - 1]) if A[» - 1] * Amin (3 . 84) 

[ Amln ifA[n-l]<A mi „ 

where A[n] is the step size at iteration (time step) n of the algorithm, and «,[»] is the 1-bit 

oerformance of the ADM algorithm based on Equation (3.84), and compare it to t 
Terf ormance of a corresponding linear delta modulator (LDM). Details of the experiment 
are as follows: 

Input signal: . 



A[n] = 



where amplitude A = 10, frequency f m = /./100, and f s = sampling frequency. 

Linear delta modulation (LDM): 

Step size A[«] = 1 for all n 

Adaptive delta modulation (ADM): 



for 



The results of the experiment are plotted in Figure 3.32. Part a of Figure , 3 32 is 
LDM, and part b of the figure is for ADM. From the waveforms presented here, we m 
make the following observations: 

► ADM tracks changes in the sinusoidal input signal muc h ^DM" ^ 

improvement in the performance of ADM is due to adaptation of the step 
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FIGURE 3,31 Adaptive delta modulation system: (a) Transmitter, (b) Receiver. 



successive iterations of the algorithm. In particular, the reduced step size of the ADM 
results in smaller quantization errors near the extremities of the input signal than the 
LDM. However, both modulation schemes produce comparable quantization errors 
in regions of the input signal where the slope is moderately high. 
The improved tracking performance of the ADM results in an output signal with a 
much lower bit rate, on the average, than the LDM. 




Figure 3.32 Waveforms resulting from the computer experiment on delta modulation: 
(a) Linear delta modulation, (b) Adaptive delta modulation. 



Chapter 3 a Pulse Modulation 



1 7 MPEG Audio Coding Standard 



Speech (voice) and audio signals are similar in that, in both cases, the quality of a coding, 
scheme is based on the properties of human auditory perception. In the case of speed, 
signals, we have efficient coding schemes (e.g., ADPCM) because a speech production 
model is available. Unfortunately, nothing similar exists for audio signals 

In this section, we revisit the MPEG-l/audio coding standard briefly described in the 
Background and Preview chapter; MPEG stands for Motion Picture Experts Group, and 
the suffix 1 is intended to mean it is the first in a series of several standards. Like ADPCM, 
the MPEG-l/audio coding standard is a lossy compression system, but it differs from 
ADPCM in an important practical respect: The MPEG-1 standard is capable of achieving 
transparent, perceptually lossless compression of stereophonic audio signals at high 
sampling rates. In particular, subjective listening tests performed by the MPEG/audio 
committee, under very difficult listening conditions, have shown that even with a 6-tol 
compression ratio, the coded and original audio signals are perceptually indistinguishable. 

The MPEG-l/audio coding standard achieves this remarkable performance by ex- 
ploiting two psychoacoustic characteristics of the human auditory system: 

1. Critical bands. . 
The inner ear 14 of the auditory system represents the power spectra of incoming signals 
on a nonlinear scale in the form of limited frequency bands called die critical bands.Va 
audible frequency band, extending up to 20 kHz, is covered by 25 critical bands, whose 
individual bandwidths increase with frequency. Loosely speaking, the auditory system may 
be modeled as a band-pass filter bank, consisting of 25 overlapping band-pass filters with 
bandwidths less than 100 Hz for the lowest audible frequencies and up to 5 kHz tor the 
highest audible frequencies. 

2. Auditory masking. . , 
Auditory masking or noise masking is a frequency-domain phenomenon that arises when 
a low-level signal (the maskee) and a high-level signal (the masker) occur simultaneously 
and are close enough to each other in frequency. If the low-level signal lies below a masking 
threshold, it is made inaudible (i.e., masked) by the stronger signal. The auditory-maskuig 
phenomenon is most pronounced when both signals lie in the same critical band, and less 
effective when they lie in neighboring bands. 

Figure 3.33 illustrates the definition of masking threshold and related parameters for 
a pair of adjacent frequency bands; it is assumed that the masker (i.e., the high-level signal) 
lies inside the dark-shaded critical band. The low-level signals lying inside this dark ate 
and below the masking threshold are masked by the stronger signal. From Figure 3 33 we 
see that the masking threshold varies with frequency across the critical band. According!), 
we may define a minimum masking threshold for a critical band, below which all low- 
level signals that lie inside that band are made inaudible by the stronger signal. The power, 
difference, expressed in decibels, between the masker and the minimum masking thresho a, 
is termed the signal-to-mask ratio (SMR). Figure 3.33 also includes the stgnaUo-noU 
ratio (SNR) for an R-bit quantizer. The difference between SMR and SNR is the notse-to. 
mask ratio (NMR) for an R-bit quantizer as shown by 

NMR = SMR - SNR < 3i5) 
where all three terms are expressed in dBs. Within a critical band, the quantization noise 
is inaudible as long as the NMR for the pertinent quantizer is negative. 
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FIGURE 333 Illustrating the definitions of masking threshold and related parameters. The high- 
level signal (masker) lies inside the darker-shaded critical band, hence the masking is more 
effective in this band than in the neighboring band shown in lighter shading. (Adapted from Noll 
(1998) with permission of the CRC Press.) 



With this background on the psychoacoustics of the auditory system, we are now 
ready to describe the operation of the MPEG-l/audio coding standard. Figure 3.34 shows 
the basic block diagrams of the encoder and decoder. The encoder consists of four func- 
tional units: time-to-frequency mapping network, psychoacoustic model, quantizer and 
coder, and frame-packing unit. The decoder consists of three functional units: frame- 
unpacking unit, frequency-sample reconstruction network, and frequency-to-time map- 
ping network. The psychoacoustic model is thus only necessary in the encoder. 

Starting with a description of the encoder first, the function of the time-to-frequency 
mapping network is to decompose the input audio signal into multiple subbands for cod- 
ing. The mapping is performed in three layers, labeled I, II, and III, which are of increasing 
complexity, delay, and subjective perceptual performance. The algorithm in layer I uses a 
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Figure 3.34 MPEG/Audio coding system, (a) Transmitter. (2i) Receiver. 
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band-pass filter bank that divides the audio signal into 32 constant-width subbands; thi s 
filter bank is also found in layers II and m. In light of our previous remarks on the no n . 
uniformly spaced critical bands, the design of this filter bank is a compromise between 
computational efficiency and perceptual performance. The algorithm in layer II is a simple 
enhancement of layer I; it improves the compression performance by coding the data in 
larger groups. Finally, the layer III algorithm is much more refined in that it is designed 
to achieve frequency resolutions closer to the partitions between the critical bands. 

The psychoacoustic model is the key component in the encoder. Its function is to 
analyze the spectral content of the input audio signal and thereby compute the signal-to. 
mask ratio for each subband in each of the three layers. This information is, in turn, used 
by the quantizer-coder to decide how to apportion the available number of bits for the 
quantization of the subband signals. This dynamic allocation of bits is performed so as to 
minimize the audibility of the quantization noise. Finally, the frame-packing unit assembles 
the quantized audio samples into a decodable bit stream. 

The decoder simply reverses the signal-processing operations performed in the en- 
coder, converting the received stream of encoded bits into a time-domain audio signal. 

To sum up, the MPEG-l/audio coding standard represents the state of the art in the 
coding of audio signals. Layer I achieves a compression ratio of 4 at an approximate stereo 
bit rate of 384 kb/s for transparent quality of performance. The corresponding compres- 
sion ratios for layers II and III are 8 and 12 at approximate stereo bit rates of 192 kb/s 
and 128 kb/s, respectively. The subjective quality of the MPEG-l/audio coding standaid 
is equivalent to compact disc quality (16-bit PCM) for many types of music; the compact 
disc (CD) is today's de facto standard of digital audio representation. 



3.18 Summary and Discussion 

In this chapter we introduced two fundamental and complementary processes: 

> Sampling, which operates in the time domain; the sampling process is the link be- 
tween an analog waveform and its discrete-time representation. 

> Quantization, which operates in the amplitude domain; the quantization process is 
the link between an analog waveform and its discrete-amplitude representation. 

The sampling process builds on the sampling theorem, which states that a strictly band- 
limited signal with no frequency components higher than W Hz is represented uniquely 
by a sequence of samples taken at a uniform rate equal to or greater than the Nyquistrate 
of 2W samples per second. The quantization process exploits the fact that any human 
sense, as ultimate receiver, can only detect finite intensity differences. 

The sampling process is basic to the operation of all pulse modulation systems, which 
may be classified into analog pulse modulation and digital pulse modulation. The distin- 
guishing feature between them is that analog pulse modulation systems maintain a contin- 
uous amplitude representation of the message signal, whereas digital pulse modulation 
systems also employ quantization to provide a representation of the message signal that is 
discrete in both time and amplitude. . , 

Analog pulse modulation results from varying some parameter of the transmitte 
pulses, such as amplitude, duration, or position, in which case we speak of pulse-amplitude 
modulation (PAM), pulse-duration modulation (PDM), or pulse-position modulation 
(PPM), respectively. In time-division multiplexing (TDM) of several channels, signal pro- 
cessing usually begins with PAM. To use PDM or PPM in such an application, we have 
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to ensure that full-scale modulation will not cause a pulse from one message signal to enter 
a time slot belonging to another message signal. This restriction results in a wasteful use 
of time space in telephone systems that are characterized by high peak factors, which is 
one reason for not using PDM or PPM in telephony. Also, despite the fact that PPM is 
more efficient than PDM, they both fall short of the ideal system for exchanging trans- 
mission bandwidth for improved noise performance. 

Digital pulse modulation systems transmit analog message signals as a sequence of 
coded pulses, which is made possible through the combined use of sampling and quanti- 
zation. Pulse-code modulation is an important form of digital pulse modulation that is 
endowed with some unique system advantages, which, in turn, have made it the standard 
method of modulation for the transmission of such analog signals as voice and video 
signals. The advantages of pulse-code modulation include robustness to noise and inter- 
ference, efficient regeneration of the coded pulses along the transmission path, and a uni- 
form format for different kinds of baseband signals. 

Indeed, it is because of this list of advantages unique to pulse-code modulation that 
it has become the method of choice for the construction of public switched telephone 
networks (PSTNs). In this context, the reader should carefully note that the telephone 
channel viewed from the PSTN to an Internet service provider, for example, is nonlinear 
due to the use of companding and, most importantly, it is entirely digital. This observation 
has a significant impact on the design of high-speed modems for communication between 
a computer user and server, as discussed in Chapter 6. 

Delta modulation and differential pulse-code modulation are two other useful forms 
of digital pulse modulation. The principal advantage of delta modulation is the simplicity 
of its circuitry. In contrast, differential pulse-code modulation employs increased circuit 
complexity to reduce channel bandwidth. The improvement is achieved by using the idea 
of prediction to remove redundant symbols from an incoming data stream. A further 
improvement in the operation of differential pulse-code modulation can be made through 
the use of adaptivity to account for statistical variations in the input data. By so doing, 
bandwidth requirement is reduced significantly without serious degradation in system 
performance. 

Unlike adaptive differential pulse-code modulation, the MPEG audio coding stan- 
dard achieves the compression of stereophonic audio signals in a transparent, perceptually 
lossless manner. This impressive performance is realized by exploiting certain psycho- 
acoustic properties of the auditory system. 

At this point in the discussion, it is informative to take a critical look at the different 
forms of pulse modulation that we have described in this chapter. In a strict sense, the 
term pulse modulation is a misnomer in that all of its different forms, be they analog or 
digital, are in fact source coding techniques. We say this for the simple reason that a 
message signal remains a baseband signal after undergoing all the changes involved in a 
pulse modulation process. The baseband nature of a pulse-modulated signal is exemplified 
by the fact that, irrespective of its exact description, it can be transmitted over a baseband 
channel of adequate bandwidth. Indeed, the material presented in the next chapter is 
devoted to the baseband transmission of data represented by a sequence of pulses. 

It is also important to recognize that pulse modulation techniques are lossy in the 
sense that some information is lost as a result of the signal representation that they perform. 
For example, in pulse-amplitude modulation, the customary practice is to use low-pass 
anti-alias filtering prior to sampling; in so doing, information is lost by virtue of the fact 
that high-frequency components considered to be unessential are removed by the filter. 
The lossy nature of pulse modulation is most vividly seen in pulse-code modulation that 
is characterized by the generation of quantization noise (i.e., distortion); the transmitted 
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sequence of encoded pulses does not have the infinite precision needed to represent con- 
tinuous samples exactly. Nevertheless, the loss of information incurred by the use of a 
pulse-modulation process is under the designer's control in that it can be made small 
enough for it to be nondiscernible by the end user. 

The material presented in this chapter on pulse modulation has been from a signal 
processing perspective. We will revisit pulse-code modulation in Chapter 9, which is de- 
voted to information-theoretic considerations of communication systems. In so doing We 
will develop deeper insight into its operation as a source coding technique. 



Notes and References 



1 The classic book on pulse modulation is Black (1953). A more detailed treatment of the 
subject is presented in the book by Rowe (1965). For the noise analysis of a PPM system, 
see the third edition of the book by Haykin (1994). 

2 Pulse-code modulation was invented by Reeves in 1937. For a historical account of this 
invention, see the paper by Reeves (1975). The book by Jayant and Noll (1984) presents 
detailed treatment of pulse-code modulation, differential pulse-code modulation, delta 
modulation, and their variants. The book edited by Jayant (1976) provides a collection of 
early papers written on waveform quantization and coding. 

3. For a detailed discussion of quantization noise in PCM systems, see the paper by Bennett 
(1948) and also the book by Rowe (1965, pp. 311-321). 

4 The two necessary conditions of Equations (3.42) and (3.47) for optimality of a scalar 
quantizer were reported independently by Lloyd (1957) and Max (1960), hence the name 
"Lloyd-Max quantizer." The derivation of these two optimality conditions presented in 
this chapter follows the book by Gersho and Gray (1992). 

5 The ii-law used for signal compression is described in Smith (1957).The ii-law is used in 
the United States, Canada, and Japan. In Europe, the A-law is used for signal compression; 
this compression law is described in Cattermode (1969, pp. 133-140). For a discussion of 
the ii-law and A-law, see also the paper by Kaneko (1970). 

6. For a description of the original version of the Tl-carrier system, see the paper by Fultz 
and Penick (1965). The description given in Example 3.2 is based on an updated version 
of this system; see Henning and Pan (1972). 

7 The North America/Japan standards for digital multiplexers were originally adopted by 
AT&T. Another set of standards has been adopted by CCITT for the rest of the world. 
The CCITT digital hierarchy is similar to that described in Section 3.10, except for certain 
changes in the specifications of the number of channel inputs to the five digital multiplexers 
and their individual bit rates. For details of the CCITT digital hierarchy, see Couch (1997). 

8. For the original papers on delta modulation, see Schouten, Dejager, and Greefkes (1952) 
and Dejager (1952). For a review paper on delta modulation, see the paper by Schindlet 
(1970). 

9. Delta-sigma modulation is described in the book by Jayant and Noll (1984, pp. 399-^00); 
see also the paper by Inose, Yasuda, and Murakami (1962). 

10. Differential pulse-code modulation was invented by Cutler; the invention is dex fxda? 
patent issued in 1 952. For a comparison of the noise performances of PCM and Dl W > 
see the paper by Jayant (1974); see also Rabiner and Schafer (1978, Chapter 5). 

11. For a discussion of adaptive differential pulse-code modulation, see Jayant and Noll <1984), 
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12. The adaptive delta modulation algorithm (ADM) of Equation (3.84) is the corrected ver- 
sion of an algorithm presented in Sklar (1988, p. 641 ). Sklar's algorithm was adapted from 
an earlier paper by Song et al. (1971), where an optimum ADM system is derived; the 
highly nonlinear equations characterizing the optimum system are approximated in the 
latter paper by piecewise-linear equations for the purpose of implementation. 

13. The MPEG-l/audio coding standard is described in the papers by Brandenburg and Stoll 
(1994), Pan (1993), and the article by Peter Noll in the handbook on Digital Signal Pro- 
cessing edited by Madisetti and Williams (1998); the latter article also discusses the follow- 
up standards to MPEG-1 . In particular, the MPEG-2 offers stereophonic audio coding at 
sampling rates lower than MPEG-1. 

14. The ear, the organ of hearing, responds to incoming acoustical waves. It has three main 
parts, with their functions as summarized here: 

^ The outer ear aids in the collection of sounds. 

> The middle ear provides an acoustic impedance match between the air and the cochlea fluids, 
thereby conveying the vibrations of the tympanic membrane (eardrum) due to the incoming 
sounds to the inner ear in an efficient manner. 

> The inner ear converts the mechanical vibrations from the middle ear to an electrochemical 
or neural signal for transmission to the brain for processing. 



Problems 

Sampling Process 

3.1 A narrowband signal has a bandwidth of 10 kHz centered on a carrier frequency of 
100 kHz. It is proposed to represent this signal in discrete-time form by sampling its in- 
phase and quadrature components individually. What is the minimum sampling rate that 
can be used for this representation? Justify your answer. How would you reconstruct the 
original narrowband signal from the sampled versions of its in-phase and quadrature 
components? 

3.2 In natural sampling, an analog signal g(t) is multiplied by a periodic train of rectangular 
pulses c(t). Given that the pulse repetition frequency of this periodic train is f s and the 
duration of each rectangular pulse is T (with f,T <K 1), do the following: 

(a) Find the spectrum of the signal s(r) that results from the use of natural sampling; you 
may assume that time t = 0 corresponds to the midpoint of a rectangular pulse in 
c(t). 

(b) Show that the original signal m(t) may be recovered exactly from its naturally sampled 
version, provided that the conditions embodied in the sampling theorem are satisfied. 

3.3 Specify the Nyquist rate and the Nyquist interval for each of the following signals: 

(a) g(t) = sinc(200f) 

(b) g{t) = sinc 2 (200t) 

(c) g{t) = sinc(200f) + sinc 2 (200r) 

3.4 (a) Plot the spectrum of a PAM wave produced by the modulating signal 

m{t) = A m cos(2-n/„() 

assuming a modulation frequency f„ = 0.25 Hz, sampling period T, = 1 s, and pulse 
duration T = 0.45 s. 

(b) Using an ideal reconstruction filter, plot the spectrum of the filter output. Compare 
this result with the output that would be obtained if there were no aperture effect. 
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Pulse-Amplitude Modulation 

3.5 Figure P3.5 shows the idealized spectrum of a message signal m(t). The signal is sampled 
at a rate equal to 1 kHz using fiat-top pulses, with each pulse being of unit amplitude 
and duration 0.1 ms. Determine and sketch the spectrum of the resulting PAM signal. 




/(Hz) 



Figure P3.5 



3.6 In this problem, we evaluate the equalization needed for the aperture effect in a PAM 
system. The operating frequency / = fjl, which corresponds to the highest frequency 
component of the message signal for a sampling rate equal to the Nyquist rate. Plot 
1/sinc (0.5T/TJ versus TIT,, and hence find the equalization needed when TIT, = 0.1. 

3.7 Consider a PAM wave transmitted through a channel with white Gaussian noise and 
minimum bandwidth B T = 1/2T„ where T s is the sampling period. The noise is of zero 
mean and power spectral density N 0 /2. The PAM signal uses a standard pulse g{t) with 
its Fourier transform defined by 



G(f) ■■ 



h 1/1 <Bt 



LO, l/l > Br 

By considering a full-load sinusoidal modulating wave, show that PAM and baseband- 
signal transmission have equal signal-to-noise ratios for the same average transmitted 
power. 

3.8 Twenty-four voice signals are sampled uniformly and then time-division multiplexed. The 
sampling operation uses flat-top samples with 1 lis duration. The multiplexing operation 
includes provision for synchronization by adding an extra pulse of sufficient amplitude 
and also 1 /xs duration. The highest frequency component of each voice signal is 3.4 kHz. 

(a) Assuming a sampling rate of 8 kHz, calculate the spacing between successive pulses 
of the multiplexed signal. 

(b) Repeat your calculation assuming the use of Nyquist rate sampling. 

3.9 Twelve different message signals, each with a bandwidth of 10 kHz, are to be multiplexed 
and transmitted. Determine the minimum bandwidth required for each method if the 
multiplexing/modulation method used is 

(a) FDM, SSB. 

(b) TDM, PAM. 

3.10 A PAM telemetry system involves the multiplexing of four input signals: s,(f), i = 1, 2, 3, 
4. Two of the signals s,(ij and s 2 (f) have bandwidths of 80 Hz each, whereas the remaining 
two signals s 3 (t) and s 4 (t) have bandwidths of 1 kHz each. The signals s 3 {t) and s 4 (t) are 
each sampled at the rate of 2400 samples per second. This sampling rate is divided by 2 
(i.e., an integer power of 2) to derive the sampling rate for s t (*) and s 2 (t). 

(a) Find the maximum value of R. 

(b) Using the value of R found in part (a), design a multiplexing system that first multi- 
plexes s^t) and s 2 (t) into a new sequence, s s {t), and then multiplexes s 3 (t), s 4 {t), and 
s s (t). 



Problems 241 



Line Codes 

3.11 In this problem we derive the formulas used to compute the power spectra of Figure 3.16 
for the five line codes described in Section 3.7. In the case of each line code, the bit 
duration is T b and the pulse amplitude A is conditioned to normalize the average power 
of the line code to unity as indicated in Figure 3.16. Assume that the data stream is 
randomly generated, and symbols 0 and 1 are equally likely. 

Derive the power spectral densities of these line codes as summarized here: 

(a) Unipolar nonreturn-to-zero signals: 

S(f) = ^sinc 2 (/T i ,)(l +jrS(f) 

(b) Polar nonreturn-to-zero signals: 

S(f) = A 2 T b sinc 2 (/T fc ) 

(c) Unipolar return-to-zero signals: 



.... A 2 T b . JfT b 
S</) = — smc 2 — 



(d) Bipolar return-to-zero signals: 

j 

' ~ 4 """" 1 2 



S(f) = ^ sincM ^ ) sinV/T.) 



(e) Manchester-encoded signals: 

S(/) = ^sinc 2 ^) sin 2 (^ 

Hence, confirm the spectral plots displayed in Figure 3.16. 

3.12 Suppose a random binary data stream (with equiprobable symbols) is differentially en- 
coded and then transmitted using one of the five line codes described in Problem 3.11. 
How is the power spectral density of the transmitted data affected by the use of differential 
encoding? Justify your answer. 

3.13 A randomly generated data stream consists of equiprobable binary symbols 0 and 1. It is 
encoded into a polar nonreturn-to-zero waveform with each binary symbol being defined 
as follows: 

*>-{-(?;)■ -?<-t 

LO, otherwise 

(a) Sketch the waveform so generated, assuming that the data stream is 00101110. 

(b) Derive an expression for the power spectral density of this signal, and sketch it. Hint: 
use Equation (1.52). 

(c) Compare the power spectral density of this random waveform with that defined in 
part (b) of Problem 3.11. 

3.14 Given the data stream 1110010100, sketch the transmitted sequence of pulses for each 
of the following fine codes: 

(a) Unipolar nonreturn-to-zero 

(b) Polar nonreturn-to-zero 

(c) Unipolar return-to-zero 

(d) Bipolar return-to-zero 

(e) Manchester code 
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3.15 Suppose the binary data stream considered in Problem 3.14 is differentially encoded aad 
then transmitted using one of the five line codes considered therein. Sketch each of the 
transmitted data streams, assuming the use of symbol 1 for the reference bit. How is the 
result affected if symbol 0 is used for the reference bit? 

Pulse-Code Modulation 

3.16 A speech signal has a total duration of 10 s. It is sampled at the rate of 8 kHz and then 
encoded. The signal-to-(quantization) noise ratio is required to be 40 dB. Calculate the 
minimum storage capacity needed to accommodate this digitized speech signal. 

3.17 Consider a uniform quantizer characterized by the input-output relation illustrated i n 
Figure 3.10a. Assume that a Gaussian-distributed random variable with zero mean and 
unit variance is applied to this quantizer input. 

(a) What is the probability that the amplitude of the input lies outside the range -4 10 
+4? 

(b) Using the result of part (a), show that the output signal-to-noise ratio of the quantizer 
is given by 

(SNR) 0 = 6R - 7.2 dB 

where R is the number of bits per sample. Specifically, you may assume that the 
quantizer input extends from -4 to +4. Compare the result of part (b) with that 
obtained in Example 3.1. 

3.18 A PCM system uses a uniform quantizer followed by a 7-bit binary encoder. The bit rate 
of the system is equal to 50 X 10 s b/s. 

(a) What is the maximum message bandwidth for which the system operates 
satisfactorily? 

(b) Determine the output signal-to-(quantization) noise ratio when a full-load sinusoidal 
modulating wave of frequency 1 MHz is applied to the input. 

3.19 Show that, with a nonuniform quantizer, the mean-square value of the quantization erroi 
is approximately equal to (1/12)2,- &}p h where A, is the rth step size and p, is the proba- 
bility that the input signal amplitude lies within the rth interval. Assume that the step size 
A, is small compared with the excursion of the input signal. 

3.20 ( a ) A sinusoidal signal, with an amplitude of 3.25 volts, is applied to a uniform quantizer 

of the midtread type whose output takes on the values 0, ±1, ±2, ±3 volts. Sketch 
the waveform of the resulting quantizer output for one complete cycle of the input, 
(b) Repeat this evaluation for the case when the quantizer is of the midrise type whose 
output takes on the values =0.5, ±1.5, ±2.5, ±3.5 volts. 

3.21 The signal 

m(t) = 6 sin{2Trt) volts 

is transmitted using a 4-bit binary PCM system. The quantizer is of the midrise type, with 
a step size of 1 volt. Sketch the resulting PCM wave for one complete cycle of the input. 
Assume a sampling rate of four samples per second, with samples taken at t = ±M>> 
±3/8, ±5/8, . . . , seconds. 

3.22 Figure P3.22 shows a PCM signal in which the amplitude levels of +1 volt and -1 volt 
are used to represent binary symbols 1 and 0, respectively. The code word used consists 
of three bits. Find the sampled version of an analog signal from which this PCM signa 
is derived. 
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Figure P3.22 



3.23 Consider a chain of (« - 1) regenerative repeaters, with a total of n sequential decisions 
made on a binary PCM wave, including the final decision made at the receiver. Assume 
that any binary symbol transmitted through the system has an independent probability 
pi of being inverted by any repeater. Let p n represent the probability that a binary symbol 
is in error after transmission through the complete system. 

(a) Show that 

Pn = |[1 " (1 ~ 2p 1 )"] 

(b) If p 1 is very small and n is not too large, what is the corresponding value of pj 

3.24 Discuss the basic issues involved in the design of a regenerative repeater for pulse-code 
modulation. 



Delta Modulation 

3.25 Consider a test signal m(t) defined by a hyperbolic tangent function: 

m(t) = A tanh(/3f) 

where A and p are constants. Determine the minimum step size A for delta modulation 
of this signal, which is required to avoid slope overload. 

3.26 Consider a sine wave of frequency f m and amplitude A„, which is applied to a delta 
modulator of step size A. Show that slope-overload distortion will occur if 



277/„T s 



where T s is the sampling period. What is the maximum power that may be transmitted 
without slope-overload distortion? 

3.27 A linear delta modulator is designed to operate on speech signals limited to 3.4 kHz. The 
specifications of the modulator are as follows: 

Sampling rate = 10/ NyqoiJt , where / Kyquist is the Nyquist rate of the speech signal. 
► Step size A = 100 mV. 

The modulator is tested with a 1-kHz sinusoidal signal. Determine the maximum ampli- 
tude of this test signal required to avoid slope overload. 

3.28 In this problem, we derive an empirical formula for the average signal-to-(quantization) 
noise ratio of a DM system with a sinusoidal signal of amplitude A and frequency f„ as 
the test signal. Assume that the power spectral density of the granular noise generated by 
the system is governed by the formula 

A 2 

where f s is the sampling rate and A is the step size. (Note that this formula is basically 
the same as that for the power spectral density of quantization noise in a PCM system 
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with A/2 for PCM being replaced by A for DM.) The DM system is designed to handle 
analog message signals limited to bandwidth W. 

(a) Show that the average quantization noise power produced by the system is 

N= 3/? 

where it is assumed that the step size A has been chosen in accordance with the 
formula used in Problem 3.27 so as to avoid slope overload. 

(b) Hence determine the signal-to-(quantization) noise ratio of the DM system for a 
sinusoidal input. 

3.29 Consider a DM system designed to accommodate analog message signals limited to band- 
width W = 5 kHz. A sinusoidal test signal of amplitude A = 1 volt and frequency 
f m = 1 kHz is applied to the system. The sampling rate of the system is 50 kHz. 

(a) Calculate the step size A required to minimize slope overload. 

(b) Calculate the signal-to-(quantization) noise ratio of the system for the specified 
sinusoidal test signal. 

For these calculations, use the formulas derived in Problems 3.27 and 3.28. 

3.30 Consider a low-pass signal with a bandwidth of 3 kHz. A linear delta modulation system, 
with step size A = 0.1V, is used to process this signal at a sampling rate ten times the 
Nyquist rate. 

(a) Evaluate the maximum amplitude of a test sinusoidal signal of frequency 1 kHz, 
which can be processed by the system without slope-overload distortion. 

(b) For the specifications given in part (a), evaluate the output signal-to-noise ratio under 
(i) prefiltered, and (ii) postfiltered conditions. 

Linear Prediction 

3.31 A one-step linear predictor operates on the sampled version of a sinusoidal signal. The 
sampling rate is equal to 10/o where f 0 is the frequency of the sinusoid. The predictor 
has a single coefficient denoted by w t . 

(a) Determine the optimum value of it>i required to minimize the prediction error 
variance. 

(b) Determine the minimum value of the prediction error variance. 

3.32 A stationary process X(f) has the following values for its autocorrelation function: 

R x (0) = 1 
RxU) = 0.8 
Rx(2) = 0.6 
Rx(3) = 0.4 

(a) Calculate the coefficients of an optimum linear predictor involving the use of three 
unit-delays. 
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involves finding the conditional probability assignment p(y ; |* ; ) subject to certain con- 
straints imposed on p(y ; |*,). This is a variational problem, the solution of which is un- 
fortunately not straightforward in general. 



P- Example 9.14 Gaussian Source 

Consider a discrete-time, memoryless Gaussian source with zero mean and variance cr 2 . Let x 
denote the value of a sample generated by such a source. Let y denote a quantized version of 
x that permits a finite representation of it. The squared error distortion 

d(x, y) = (x - yf 

provides a distortion measure that is widely used for continuous alphabets. The rate distortion 
function for the Gaussian source with squared error distortion, as described herein, is given 
by 



\\ log 0 £ D £ <t 2 

R(D) = -U B \Dj' (9.133) 

L0, D > o- 2 

In this case, we see that R(D) -» ■» as D -* 0, and R(D) = 0 for D = cr 2 . < 
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Figure 9.23 Reverse water-filling picture for a set of parallel Gaussian processes. 



Example 9.15 Set of Parallel Gaussian Sources 

Consider next a set of N independent Gaussian random variables (X,},5i, where X, has zero 
mean and variance of. Using the distortion measure 

d = 2 <*, - 

i- 1 

and building on the result of Example 9.14, we may express the rate distortion function for 
the set of parallel Gaussian sources described here as 



R(D) ■■ 



U l0S (l, 



D, 



where D, is itself defined by 

[A if A < <7, 
[of if A > of 
and the constant A is chosen so as to satisfy the condition 

N 

2 D, = D 



(9.134) 



(9.135) 



(9.136) 



Equations (9.135) and (9.136) may be interpreted as a kind of "water-filling in reverse," as 
illustrated in Figure 9.23. First, we choose a constant A and only the subset of random variables 
whose variances exceed the constant A. No bits are used to describe the remaining subset of 
random variables whose variances are less than the constant A. ^ 



1 9.14 Data Compression 

Rate distortion theory naturally leads us to consider the idea of data compression that 
involves a purposeful or unavoidable reduction in the information content of data from a 
continuous or discrete source. Specifically, we may think of a data compressor, or signal 
compressor, as a device that supplies a code with the least number of symbols for the 
representation of the source output, subject to a permissible or acceptable distortion. The 
data compressor thus retains the essential information content of the source output by 
blurring fine details in a deliberate but controlled manner. Accordingly, data compression 
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is a lossy operation in the sense that the source entropy is reduced (i.e., information is 
lost), irrespective of the type of source being considered. 

In the case of a discrete source, the reason for using data compression is to encode 
the source output at a rate smaller than the source entropy. By so doing, the source coding 
theorem is violated, which means that exact reproduction of the original data is no longer 
possible. 

In the case of a continuous source, the entropy is infinite, and therefore a signal 
compression code must always be used to encode the source output at a finite rate. Con- 
sequently, it is impossible to digitally encode an analog signal with a finite number of bits 
without producing some distortion. This statement is in perfect accord with the idea of 
pulse-code modulation, which was studied in Chapter 3. There it was shown that quan- 
tization, which is basic to the analog-to-digital conversion process in pulse-code modula- 
tion, always introduces distortion (known as quantization noise) into the transmitted sig- 
nal. A quantizer may therefore be viewed as a signal compressor. 

The uniform and nonuniform quantizers considered in Chapter 3 are said to be scalar 
quantizers in the sense that they deal with samples of the analog signal (i.e., continuous 
source output) one at a time. Each sample is converted into a quantized value, with the 
conversion being independent from sample to sample. A scalar quantizer is a rather simple 
signal compressor, which makes it attractive for practical use. Yet it can provide a sur- 
prisingly good performance; this is especially so if nonuniform quantization is used. 

There is another class of quantizers known as vector quantizers that use blocks of 
consecutive samples of the source output to form vectors, each of which is treated as a 
single entity. The essential operation in a vector quantizer is the quantization of a random 
vector 16 by encoding it as a binary code word. The vector is encoded by comparing it with 
a codebook consisting of a set of stored reference vectors known as code vectors or pat- 
terns. Each pattern in the codebook is used to represent input vectors that are identified 
by the encoder to be similar to the particular pattern, subject to the maximization of an 
appropriate fidelity criterion. The encoding process in a vector quantizer may thus be 
viewed as a pattern matching operation. 

Let N be the number of code vectors in the codebook, k be the dimension of each 
vector (i.e., the number of samples in each pattern), and r be the coded transmission rate 
in bits per sample. These three parameters are related as follows: 



Then, assuming that the size of the code book is sufficiently large, the signal-to-quanti- 
zation noise ratio (SNR) for the vector quantizer is given by 



where C k is a constant (expressed in dB) that depends on the dimensions k. According to 
Equation (9.138), the SNR for a vector quantizer increases approximately at the rate of 
61k dB for each doubling of the codebook size. Equivalently, we may state that the SNR 
increases by 6 dB per unit increase in rate (bits per sample) as in the standard PCM using 
a uniform scalar quantizer. The advantage of the vector quantizer over the scalar quantizer 
is that its constant term C k has a higher value, because the vector quantizer optimally 
exploits the correlations among the samples constituting a vector. Specifically, the constant 
C k increases with the dimension k, approaching the ultimate rate-distortion limit for a 




(9.138) 
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given source of information. However, the improvement in SNR is attained at the cost of 
increased encoding complexity, which grows exponentially with the dimension k for a 
specified rate r. Unfortunately, this is the main obstacle to the wide use of vector quanti- 
zation in practice. Nevertheless, in certain applications, the issue of computational com- 
plexity is mitigated by exploiting the capability of VLSI technology to concentrate a highly 
complex signal processor on a silicon chip. For example, that is precisely what is done in 
the use of code-excited linear predictive (CELP) modeling of speech in wireless commu- 
nication systems of the CDMA type, namely, the IS-95 system. From the description of 
CELP presented in Section 8.9, it is clear that the CELP modeling of speech is an example 
of vector quantization. 



I 9 . 1 5 Summary and Discussion 

In this chapter we established four fundamental limits on different aspects of a commu- 
nication system. The limits are embodied in the source coding theorem, the channel coding 
theorem, the information capacity theorem, and the rate distortion function. 

The source coding theorem, Shannon's first theorem, provides the mathematical tool 
for assessing data compaction, that is, lossless compression of data generated by a discrete 
memoryless source. The theorem tells us that we can make the average number of binary 
code elements (bits) per source symbol as small as, but no smaller than, the entropy of the 
source measured in bits. The entropy of a source is a function of the probabilities of the 
source symbols that constitute the alphabet of the source. Since entropy is a measure of 
uncertainty, the entropy is maximum when the associated probability distribution gener- 
ates maximum uncertainty. 

The channel coding theorem, Shannon's second theorem, is both the most surprising 
and the single most important result of information theory. For a binary symmetric chan- 
nel, the channel coding theorem tells us that for any code rate r less than or equal to the 
channel capacity C, codes do exist such that the average probability of error is as small as 
we want it. A binary symmetric channel is the simplest form of a discrete memoryless 
channel. It is symmetric because the probability of receiving a 1 if a 0 is sent is the same 
as the probability of receiving a 0 if a 1 is sent. This probability, the probability that an 
error will occur, is termed a transition probability. The transition probability p is deter- 
mined not only by the additive noise at the channel output but also by the kind of receiver 
used. The value of p uniquely defines the channel capacity C. 

Shannon's third remarkable theorem, the information capacity theorem, tells us that 
there is a maximum to the rate at which any communication system can operate reliably 
(i.e., free of errors) when the system is constrained in power. This maximum rate is called 
the information capacity, measured in bits per second. When the system operates at a rate 
greater than the information capacity, it is condemned to a high probability of error, 
regardless of the choice of signal set used for transmission or the receiver used for pro- 
cessing the received signal. 

Finally, the rate distortion function provides the mathematical tool for signal com- 
pression (i.e., solving the problem of source coding with a fidelity criterion): The rate 
distortion function can be applied to a discrete as well as continuous memoryless source. 

When the output of a source of information is compressed in a lossless manner, the 
resulting data stream usually contains redundant bits. These redundant bits can be removed 
by using a lossless algorithm such as Huffman coding or the Lempel-Ziv algorithm for 
data compaction. We may thus speak of data compression followed by data compaction 
as two constituents of the dissection of source coding, which is so called because it refers 
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exclusively to the sources of information. In some source coding applications, we have a 
third constituent, namely, data encryption, which follows data compaction. The purpose 
of data encryption is to disguise the data (bit) stream in such a way that it has no meaning 
to an unauthorized receiver. Some basic aspects of cryptography, which encompasses both 
encryption and decryption, follow quite naturally from information theory, as discussed 
in Appendix 5. Other issues relating to cryptography are also discussed in that appendix. 

One last comment is in order. Shannon's information theory, as presented in this 
chapter, has been entirely in the context of memoryless sources and channels. The theory 
can be extended to deal with sources and channels with memory, in which case a symbol 
of interest depends on preceding symbols; however, the level of exposition needed to do 
this is beyond the scope of this book. 17 



Notes and References 



1. According to Lucky (1 989), the first mention of the term information theory by Shannon 
occurs in a 1 945 memorandum entitled "A Mathematical Theory of Cryptography." It is 
rather curious that the term was never used in the classic 1948 paper by Shannon, which 
laid down the foundations of information theory. For an introductory treatment of infor- 
mation theory, see Chapter 2 of Lucky (1989) and the paper by Wyner (1981); see also 
the books of Adamek (1991), Hamming (1980), and Abramson (1963). For more advanced 
treatments of the subject, see the books of Cover and Thomas (1991), Blahut (1987), 
McEliece (1977), and Gallager (1968). For a collection of papers on the development of 
information theory (including the 1948 classic paper by Shannon), see Slepian (1974). For 
a collection of the papers published by Shannon, see Sloane and Wyner (1993). 

2. The use of a logarithmic measure of information was first suggested by Hardey (1928); 
however, Hartley used logarithms to base 10. 

3. In statistical physics, the entropy of a physical system is defined by (Reif, 1967, p. 147) 

<f = k log a 

where k is Boltzmann's constant, O is the number of states accessible to the system, and 
log denotes the natural logarithm. This entropy has the dimensions of energy because its 
definition involves the constant k. In particular, it provides a quantitative measure of the 
degree of randomness of the system. Comparing the entropy of statistical physics with that 
of information theory, we see that they have a similar form. For a detailed discussion of 
the relation between them, see Pierce (1961, pp. 184-207) and Brillouin (1962). 

4. For the original proof of the source coding theorem, see Shannon (1948). A general proof 
of the source coding theorem is also given in the following books: Viterbi and Omura 
(1979, pp. 13-19), McEliece (1977, Chapter 3), and Gallager (1968, pp. 38-55). The 
source coding theorem is also referred to in the literature as the noiseless coding theorem, 
noiseless in the sense that it establishes the condition for error-free encoding to be possible. 

5. For proof of the Kraft-McMillan inequality, see Cover and Thomas (1991, pp. 82-84), 
Blahut (1990, pp. 298-299), and McEliece (1977, pp. 239-240). For a proof of Equation 
(9.23), see Cover and Thomas (1991), pp. 87-88), Blahut (1990, pp. 300-301), and 
McEliece (1977, pp. 241-242). 

6. The Huffman code is named after its inventor: D. A. Huffman (1952). For a readable 
account of Huffman coding and its use in data compaction, see Adamek (1991). 

7. The original papers on the Lempel-Ziv algorithm are Ziv and Lempel (1977, 1978). For 
readable descriptions of the Lempel-Ziv algorithm, see Lucky (1989, pp. 118-122), Blahut 
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(1990, pp. 314-319), and Gitlin, Hayes, and Weinstein (1992, pp. 120-122). For the 
application of the Lempel-Ziv algorithm to the compaction of English text, see Lucky 
(1989, pp. 122-128) and the paper by Welch (1984); see also the review paper by Weiss 
and Shremp (1993). 

8. The channel coding theorem is also known as the noisy coding theorem. The original proof 
of the theorem is given in Shannon (1948). A proof of the theorem is also presented in 
Hamming (1980, Chapters 9 and 10) in sufficient detail so that a general appreciation of 
relevant results is developed. The second part of the theorem is referred to in the literature 
as the converse to the coding theorem. A proof of this theorem is presented in the following 
references: Viterbi and Omura (1979, pp. 28-34) and Gallager (1968, pp. 76-82). 

9. The quantity 

/>*>lo g2 (£fj)^ 

on the left-hand side of Equation (9.70) is called relative entropy or the Kullback-Leibler 
divergence between the probability density functions f x {x) and f Y (x); see Kullback (1968). 

10. Shannon's information capacity theorem is also referred to in the literature as the Shannon- 
Hartley law in recognition of early work by Hartley on information transmission (Hartley, 
1928). In particular, Hartley showed that the amount of information that can be trans- 
mitted over a given channel is proportional to the product of the channel bandwidth and 
the time of operation. 

11. A lucid exposition of sphere packing is presented in Cover and Thomas (1991, pp. 242- 
243); see also Wozencraft and Jacobs (1965, pp. 323-341). 

12. Parts a and b of Figure 9.18 follow the corresponding parts of Figure 6.2 in the book by 
Frey (1998). 

13. For a rigorous treatment of the information capacity of a colored noisy channel, see Gal- 
lager (1968). The idea of replacing the channel model of Figure 9.19a with that of Figure 
9.19fc is discussed in Gitlin, Hayes, and Weinstein (1992). 

14. For a complete treatment of rate distortion theory, see the book by Berger (1971); this 
subject is also treated in somewhat less detail in Cover and Thomas (1991), McEliece 
(1977), and Gallager (1968). 

15. For the derivation of Equation (9.131), see Cover and Thomas (1991, p. 345). An algorithm 
for computation of the rate distortion function R(D) defined in Equation (9.131) is de- 
scribed in Blahut (1987, pp. 220-221) and Cover and Thomas (1991, pp. 364-367). 

16. For the early papers on vector quantization, see Gersho (1979) and Linde, Buzo, and Gray 
(1980). For a tutorial review of vector quantization, see Gray (1984). Equation (9.138), 
defining the SNR for a vector quantizer, is discussed in Gersho and Cuperman (1983). For 
a complete treatment of vector quantization, see the book by Gersho and Gray (1992). 

17. For detailed discussion of discrete channels with memory, see Gallager (1968, pp. 97-112) 
and Ash (1965, pp. 211-229). 



Problems 



Entropy 

9.1 Let p denote the probability of some event. Plot the amount of information gained by the 
occurrence of this event for 0 S p s 1. 
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9.2 A source emits one of four possible symbols during each signaling interval. The symbols 
occur with the probabilities: 

Po = 0.4 
p, = 0.3 
Pi = 0.2 
P, = 0.1 

Find the amount of information gained by observing the source emitting each of these 
symbols. 

9.3 A source emits one of four symbols s 0 , s ly s 2 , and s 3 with probabilities 1/3, 1/6, 1/4, and 
1/4, respectively. The successive symbols emitted by the source are statistically indepen- 
dent. Calculate the entropy of the source. 

9.4 Let X represent the outcome of a single roll of a fair die. What is the entropy of X? 

9.5 The sample function of a Gaussian process of zero mean and unit variance is uniformly 
sampled and then applied to a uniform quantizer having the input-output amplitude char- 
acteristic shown in Figure P9.5. Calculate the entropy of the quantizer output. 
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Figure P9.5 



9.6 Consider a discrete memoryless source with source alphabet Sf = {s 0 , s ly . . . , Sk_i) and 
source statistics [p a , p u . . . , pn-i}- The «th extension of this source is another discrete 
memoryless source with source alphabet 9™ = {cr 0 , a lt . . . , (Tm-i}-, where M = K". Let 
P(<jj) denote the probability of a-,-. 

(a) Show that 

M-1 

2 P("v) = i 

which is to be expected. 

(b) Show that 

2 iV.) log J ^- = Hm, k = l,2,...,n 

where p ik is the probability of symbol s- v and H(Sf) is the entropy of the original 
source. 

(c) Hence, show that 

M-1 

H(sn = 2 PMiofo — 
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9.7 Consider a discrete memoryless source with source alphabet = {s 0 , Si, s 2 ) and source 
statistics {0.7, 0.15, 0.15). 

(a) Calculate the entropy of the source. 

(b) Calculate the entropy of the second-order extension of the source. 

9.8 It may come as a surprise, but the number of bits needed to store text is much less than 
that required to store its spoken equivalent. Can you explain the reason for it? 

Data Compaction 

9.9 Consider a discrete memoryless source whose alphabet consists of K equiprobable 
symbols. 

(a) Explain why the use of a fixed-length code for the representation of such a source is 
about as efficient as any code can be. 

(b) What conditions have to be satisfied by K and the code-word length for the coding 
efficiency to be 100 percent? 

9.10 Consider the four codes listed below: 



Symbol 


Code I 


Code II 


Code III 


Code IV 


So 


0 


0 


0 


00 


H 


10 


01 


01 


01 


Sz 


110 


001 


011 


10 


s 3 


1110 


0010 


110 


110 


5 4 


1111 


0011 


111 


111 



(a) Two of these four codes are prefix codes. Identify them, and construct their individual 
decision trees. 

(b) Apply the Kraft-McMillan inequality to codes I, II, III, and IV. Discuss your results 
in light of those obtained in part (a). 

9.11 Consider a sequence of leters of the English alphabet with their probabilities of occurrence 
as given here: 

Letter a i 1 m n o p y 

Probability 0.1 0.1 0.2 0.1 0.1 0.2 0.1 0.1 

Compute two different Huffman codes for this alphabet. In one case, move a combined 
symbol in the coding procedure as high as possible, and in the second case, move it as 
low as possible. Hence, for each of the two codes, find the average code-word length and 
the variance of the average code-word length over the ensemble of letters. 

9.12 A discrete memoryless source has an alphabet of seven symbols whose probabilities of 
occurrence are as described here: 

Symbol s 0 s t s 2 s 3 s 4 s 5 s 6 

Probability 0.25 0.25 0.125 0.125 0.125 0.0625 0.0625 

Compute the Huffman code for this source, moving a "combined" symbol as high as 
possible. Explain why the computed source code has an efficiency of 100 percent. 

9.13 Consider a discrete memoryless source with alphabet {s 0 , s 1s s 2 ] and statistics {0.7, 0.15, 
0.15) for its output. 

(a) Apply the Huffman algorithm to this source. Hence, show that the average code- 
word length of the Huffman code equals 1.3 bits/symbol. 
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(b) Let the source be extended to order two. Apply the Huffman algorithm to the resulting 
extended source, and show that the average code-word length of the new code equals 
1.1975 bits/symbol. 

(c) Compare the average code-word length calculated in part (b) with the entropy of the 
original source. 

9.14 Figure P9.14 shows a Huffman tree. What is the code word for each of the symbols A, 
B, C, D, E, F, and G represented by this Huffman tree? What are their individual code- 
word lengths? 
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Figure P9.14 



9.15 A computer executes four instructions that are designated by the code words 
(00, 01, 10, 11). Assuming that the instructions are used independently with probabilities 
(1/2, 1/8, 1/8, 1/4), calculate the percentage by which the number of bits used for the 
instructions may be reduced by the use of an optimum source code. Construct a Huffman 
code to realize the reduction. 

9.16 Consider the following binary sequence 

11101001100010110100 . . . 

Use the Lempel-Ziv algorithm to encode this sequence. Assume that the binary symbols 
0 and 1 are already in the codebook. 

Binary Symmetric Channel 

9.17 Consider the transition probability diagram of a binary symmetric channel shown in 
Figure 9.8. The input binary symbols 0 and 1 occur with equal probability. Find the 
probabilities of the binary symbols 0 and 1 appearing at the channel output. 

9.18 Repeat the calculation in Problem 9.17, assuming that the input binary symbols 0 and 1 
occur with probabilities 1/4 and 3/4, respectively. 

Mutual Information and Channel Capacity 

9.19 Consider a binary symmetric channel characterized by the transition probability p. Plot 
the mutual information of the channel as a function of p lt the a priori probability of 
symbol 1 at the channel input; do your calculations for the transition probability p = 0, 
0.1, 0.2, 0.3, 0.5. 



Chapter 9 b Fundamental Limits in Information Theory 



9.20 Figure 9.10 depicts the variation of the channel capacity of a binary symmetric chan 1 
with the transition probability p. Use the results of Problem 9.19 to explain this variation 

9.21 Consider the binary symmetric channel described in Figure 9.8. Let p 0 denote the p ro b 
ability of sending binary symbol x 0 = 0, and let p, = 1 - p 0 denote the probability 0 f 
sending binary symbol x x = 1. Let p denote the transition probability of the channel ° 

(a) Show that the mutual information between the channel input and channel output is 
given by 



t; <3/) = 3C(z) - W(p) 



where 



and 



H(z) = z log 2 (^-J + (1 - z) log 2 
z = p 0 p + (1 - p 0 )(l - p) 



H(p) = p log 2 



+ (l-p). 



(b) Show that the value of p 0 that maximizes % is equal to 1/2. 

(c) Hence, show that the channel capacity equals 

C = 1 - H(p) 

9.22 Two binary symmetric channels are connected in cascade, as shown in Figure P9.22. Find 
the overall channel capacity of the cascaded connection, assuming that both channels 
have the same transition probability diagram shown in Figure 9.8. 
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Figure P9.22 



9.23 The binary erasure channel has two inputs and three outputs as described in Figure P9.23. 
The inputs are labeled 0 and I, and the outputs are labeled 0, 1, and e. A fraction a of 
the incoming bits are erased by the channel. Find the capacity of the channel. 




1 -a 

Figure P9.23 
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9.24 Consider a digital communication system that uses a repetition code for the channel 
encoding/decoding. In particular, each transmission is repeated n times, where n = 2m + 
1 is an odd integer. The decoder operates as follows. If in a block of n received bits, the 
number of Os exceeds the number of Is, the decoder decides in favor of a 0. Otherwise, 
it decides in favor of a 1. An error occurs when m + 1 or more transmissions out of n = 
2m + 1 are incorrect. Assume a binary symmetric channel. 

(a) For n = 3, show that the average probability of error is given by 

p. = 3 P 2 (i - p) +y 

where p is the transition probability of the channel. 

(b) For n - 5, show that the average probability of error is given by 

P e = 10 ? 3 (1 - pf + 5p 4 (l -p) + p 5 

(c) Hence, for the general case, deduce that the average probability of error is given by 

Differential Entropy 

9.25 Let X,, X 2> . . . , X„ denote the elements of a Gaussian vector X. The X f are independent 
with mean ft, and variance of, i = 1, 2, . . . , n. Show that the differential entropy of the 
vector X equals 

*(X) = j lo gl [2ne(*W2 ■ . . oif"] 

What does h{X) reduce to if the variances are equal? 

9.26 A continuous random variable X is constrained to a peak magnitude M; that is, 
-M < X < M. 

(a) Show that the differential entropy of X is maximum when it is uniformly distributed, 
as shown by 

f w _ | 1/2M » -M < x < M 
[0, otherwise 

(b) Show that the maximum differential entropy of X is log 2 2M. 

9.27 Prove the properties given in Equations (9.79) to (9.81) for the mutual information 
KX; Y). 

9.28 Consider the continuous random variable Y defined by 

Y = X + N 

where X and N are statistically independent. Show that the conditional differential en- 
tropy of Y, given X, equals 

b(Y\X) = h(N) 
where h(N) is the differential entropy of N. 

Information Capacity 

9.29 A voice-grade channel of the telephone network has a bandwidth of 3.4 kHz. 

(a) Calculate the information capacity of the telephone channel for a signal-to-noise ratio 
of 30 dB. 

(b) Calculate the minimum signal-to-noise ratio required to support information trans- 
mission through the telephone channel at the rate of 9,600 b/s. 
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9.30 Alphanumeric data are entered into a computer from a remote terminal through a voice- 
grade telephone channel. The channel has a bandwidth of 3.4 kHz and output signal-to- 
noise ratio of 20 dB. The terminal has a total of 128 symbols. Assume that the symbols 
are equiprobable and the successive transmissions are statistically independent. 

(a) Calculate the information capacity of the channel. 

(b) Calculate the maximum symbol rate for which error-free transmission over the chan- 
nel is possible. 

9.31 A black-and-white television picture may be viewed as consisting of approximately 
3 X 10 5 elements, each of which may occupy one of 10 distinct brightness levels with 
equal proba bility. Assume that ( 1 ) the rate of transmission is 3 0 picture frames per second 
and (2) the signal-to-noise ratio is 30 dB. 

Using the information capacity theorem, calculate the minimum bandwidth re- 
quired to support the transmission of the resulting video signal. 

Note: As a matter of interest, commercial television transmissions actually employ a band- 
width of 4.2 MHz, which fits into an allocated bandwidth of 6 MHz. 

9.32 In this problem, we continue with Example 9.9. Suppose that the tightly packed constel- 
lation of Figure 9.15b is scaled upward so that the transmitted signal energy per symbol 
is maintained at the same average value as that consumed by the 64-QAM square con- 
stellation of Figure 9.15a. Construct the new constellation that results from this scaling. 
How does the bit error rate of this new constellation compare with that of Figure 9.15a? 
Justify your answer. 

9.33 The squared magnitude response of a twisted-pair channel can be modeled as 

|H(/)| 2 = exp(-aV/) 

The constant a is defined by 

kl 

a = j 
h 

where k is a constant depending on wire gauge, l 0 is a reference line length, and / is the 
actual length of the twisted pair under study. The squared magnitude response of the 
coupling responsible for NEXT has the form 

IHkextI/)! 2 = Pf 2 
where p is a constant that depends on the type of cable used. 

Formulate the expression for the information capacity of the NEXT-dominated 
channel described here. 

Data Compression 

9.34 Equation (9.138) for the signal-to-noise ratio (SNR) of a vector quantizer includes the 
SNR formula of Equation (3.33) for standard pulse-code modulation as a special case for 
which k = 1. Justify the validity of this inclusion. 

9.35 All practical data compression and data transmission schemes lie between two limits set 
by the rate distortion function and the channel capacity theorem. Both of these theorems 
involve the notion of mutual information, but in different ways. Elaborate on the issues 
raised by these two statements. 

Computer Experiment 

9.36 In this problem, we revisit Example 9.12, which deals with coded binary antipodal sig- 
naling over an additive white Gaussian noise (AWGN) channel. Starting with Equation 
(9.112) and the underlying theory, develop a software package for computing the mini- 
mum £f,/N„ required for a given bit error rate, where E b is the signal energy per bit, and 
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Figure P9.36 



N 0 /2 is the noise spectral density. Hence, compute the results plotted in parts a and b of 
Figure 9.18. 

As mentioned in Example 9.12, the computation of the mutual information between 
the channel input and channel output is well approximated using Monte Carlo integra- 
tion. To explain how this method works, consider a function g(y) that is difficult to sample 
randomly, which is indeed the case for the problem at hand. (For our problem, the func- 
tion g(y) represents the complicated integrand in the formula for the differential entropy 
of the channel output.) For the computation, proceed as follows: 
► Find an area A that includes the region of interest and that is easily sampled. 
*~ Choose N points, uniformly randomly inside the area A. 

Then the Monte Carlo integration theorem states that the integral of the function g(y) 
with respect to y is approximately equal to the area A multiplied by the fraction of points 
that reside below the curve of g, as illustrated in Figure P9.36. The accuracy of the ap- 
proximation improves with increasing N. 



Error-Control 
Coding 



This chapter is the natural sequel to the preceding chapter on Shannon's information 
theory. In particular, in this chapter we present error-control coding techniques that 
provide different ways of implementing Shannon's channel-coding theorem. Each error- 
control coding technique involves the use of a channel encoder in the transmitter and a 
decoding algorithm in the receiver. 

The error-control coding techniques described herein include the following important 
classes of codes: 

► Linear block codes. 

► Cyclic codes. 

► Convolutional codes. 

► Compound codes exemplified by turbo codes and low-density parity-check codes, and 
their irregular variants. 



1 Ot 1 Introduction 



The task facing the designer of a digital communication system is that of providing a cost- 
effective facility for transmitting information from one end of the system at a rate and a 
level of reliability and quality that are acceptable to a user at the other end. The two key 
system parameters available to the designer are transmitted signal power and channel 
bandwidth. These two parameters, together with the power spectral density of receiver 
noise, determine the signal energy per bit-to-noise power spectral density ratio E/,/N 0 . In 
Chapter 6, we showed that this ratio uniquely determines the bit error rate for a particular 
modulation scheme. Practical considerations usually place a limit on the value that we can 
assign to E b IN 0 . Accordingly, in practice, we often arrive at a modulation scheme and find 
that it is not possible to provide acceptable data quality (i.e., low enough error perfor- 
mance). For a fixed E b /N 0 , the only practical option available for changing data quality 
from problematic to acceptable is to use error-control coding. 

Another practical motivation for the use of coding is to reduce the required £j,/No 
for a fixed bit error rate. This reduction in E(,/N 0 may, in turn, be exploited to reduce the 
required transmitted power or reduce the hardware costs by requiring a smaller antenna 
size in the case of radio communications. 

Error control 1 for data integrity may be exercised by means of forward error cor- 
rection (FEC). Figure 10.1a shows the model of a digital communication system using such 
an approach. The discrete source generates information in the form of binary symbols. 
The channel encoder in the transmitter accepts message bits and adds redundancy accord- 
ing to a prescribed rule, thereby producing encoded data at a higher bit rate. The channel 



626 



10.1 Introduction 62 7 



Discrete channel 



Discrete 




Channel 




Modulator 




Waveform 




Detector 


1 

1 


Channel 




User 


source 




encoder 






channel 




I 
i 


decoder 





Noise 

(a) 



Discrete 




Encoder/modulator 




Waveform 




Detector/decoder 




User 


source 






channel 







Noise 

(M 

Figure 10. 1 Simplified models of digital communication system, (a) Coding and modulation 
performed separately, (b) Coding and modulation combined. 



decoder in the receiver exploits the redundancy to decide which message bits were actually 
transmitted. The combined goal of the channel encoder and decoder is to minimize the 
effect of channel noise. That is, the number of errors between the channel encoder input 
(derived from the source) and the channel decoder output (delivered to the user) is 
minimized. 

For a fixed modulation scheme, the addition of redundancy in the coded messages 
implies the need for increased transmission bandwidth. Moreover, the use of error-control 
coding adds complexity to the system, especially for the implementation of decoding op- 
erations in the receiver. Thus, the design trade-offs in the use of error-control coding to 
achieve acceptable error performance include considerations of bandwidth and system 
complexity. 

There are many different error-correcting codes (with roots in diverse mathematical 
disciplines) that we can use. Historically, these codes have been classified into block codes 
and convolutional codes. The distinguishing feature for this particular classification is the 
presence or absence of memory in the encoders for the two codes. 

To generate an (n, k) block code, the channel encoder accepts information in suc- 
cessive 4-bit blocks; for each block, it adds n - k redundant bits that are algebraically 
related to the k message bits, thereby producing an overall encoded block of n bits, where 
n > k. The «-bit block is called a code word, and n is called the block length of the code. 
The channel encoder produces bits at the rate R 0 = {nlk)R s , where R s is the bit rate of the 
information source. The dimensionless ratio r = kin is called the code rate, where 
0 < r < 1. The bit rate R 0 , coming out of the encoder, is called the channel data rate. 
Thus, the code rate is a dimensionless ratio, whereas the data rate produced by the source 
and the channel data rate are both measured in bits per second. 

In a convolutional code, the encoding operation may be viewed as the discrete- 
time convolution of the input sequence with the impulse response of the encoder. The 
duration of the impulse response equals the memory of the encoder. Accordingly, the 
encoder for a convolutional code operates on the incoming message sequence, using 
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a "sliding window" equal in duration to its own memory. This, in turn, means that in 
a convolutional code, unlike a block code, the channel encoder accepts message bits as a 
continuous sequence and thereby generates a continuous sequence of encoded bits at 
a higher rate. 

In the model depicted in Figure 10.1a, the operations of channel coding and modu- 
lation are performed separately in the transmitter; likewise for the operations of detection 
and decoding in the receiver. When, however, bandwidth efficiency is of major concern, 
the most effective method of implementing forward error-control correction coding is to 
combine it with modulation as a single function, as shown in Figure 10.1b. In such an 
approach, coding is redefined as a process of imposing certain patterns on the transmitted 
signal. 

a Automatic-Repeat Request 

Feed-forward error correction (FEC) relies on the controlled use of redundancy in the 
transmitted code word for both the detection and correction of errors incurred during 
the course of transmission over a noisy channel. Irrespective of whether the decoding of 
the received code word is successful, no further processing is performed at the receiver. 
Accordingly, channel coding techniques suitable for FEC require only a one-way link be- 
tween the transmitter and receiver. 

There is another approach known as automatic-repeat request (ARQ) for solving 
the error-control problem. The underlying philosophy of ARQ is quite different from that 
of FEC. Specifically, ARQ uses redundancy merely for the purpose of error detection. Upon 
the detection of an error in a transmitted code word, the receiver requests a repeat trans- 
mission of the corrupted code word, which necessitates the use of a return path (i.e., a 
feedback channel). As such, ARQ can be used only on half-duplex or full-duplex links. In 
a half-duplex link, data transmission over the link can be made in either direction but not 
simultaneously. On the other hand, in a full-duplex link, it is possible for data transmission 
to proceed over the link in both directions simultaneously. 

A half-duplex link uses the simplest ARQ scheme known as the stop-and-wait strat- 
egy. In this approach, a block of message bits is encoded into a code word and transmitted 
over the channel. The transmitter then stops and waits for feedback from the receiver. The 
feedback signal can be acknowledgment of a correct receipt of the code word or a request 
for transmission of the code word because of an error in its decoding. In the latter case, 
the transmitter resends the code word in question before moving onto the next block of 
message bits. 

The idling problem in stop-and-wait ARQ results in reduced data throughput, which 
is alleviated in another type of ARQ known as continuous ARQ with pullback. This second 
strategy uses a full-duplex link, thereby permitting the receiver to send a feedback signal 
while the transmitter is engaged in sending code words over the forward channel. Specif- 
ically, the transmitter continues to send a succession of code words until it receives a 
request from the receiver (on the feedback channel) for a retransmission. At that point, 
the transmitter stops, pulls back to the particular code word that was not decoded correctly 
by the receiver, and retransmits the complete sequence of code words starting with the 
corrupted one. 

In a refined version of continuous ARQ known as the continuous ARQ with selective 
repeat, data throughout is improved further by only retransmitting the code word that, 
was received with detected errors. In other words, the need for retransmitting the success- 
fully received code words following the corrupted code word is eliminated. 
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The three types of ARQ described here offer trade-offs of their own between the 
need for a half-duplex or full-duplex link and the requirement for efficient use of com- 
munication resources. In any event, they all rely on two premises: 

*■ Error detection, which makes the design of the decoder relatively simple. 
*■ Noiseless feedback channel, which is not a severe restriction because the rate of 
information flow over the feedback channel is typically quite low. 

For these reasons, ARQ is widely used in computer-communication systems. 

Nevertheless, the fact that EEC requires only one-way links for its operation makes 
the FEC much wider in application than ARQ. Moreover, the increased decoding com- 
plexity of FEC due to the combined need for error detection and correction is no longer 
a pressing practical issue because the decoder usually lends itself to microprocessor or 
VLSI implementation in a cost-effective manner. 



1 10.2 Discrete -Memoryless Channels 

Returning to the model of Figure 10.1a, the waveform channel is said to be memoryless 
if the detector output in a given interval depends only on the signal transmitted in that 
interval, and not on any previous transmission. Under this condition, we may model the 
combination of the modulator, the waveform channel, and the detector as a discrete 
memoryless channel. Such a channel is completely described by the set of transition prob- 
abilities p{j\i), where i denotes a modulator input symbol, ;' denotes a demodulator output 
symbol, and p(j\i) denotes the probability of receiving symbol /, given that symbol i was 
sent. (Discrete memoryless channels were described previously at some length in Section 
9.5.) 

The simplest discrete memoryless channel results from the use of binary input and 
binary output symbols. When binary coding is used, the modulator has only the binary 
symbols 0 and 1 as inputs. Likewise, the decoder has only binary inputs if binary quan- 
tization of the demodulator output is used, that is, a hard decision is made on the demod- 
ulator output as to which symbol was actually transmitted. In this situation, we have a 
binary symmetric channel (BSC) with a transition probability diagram as shown in Figure 
10.2. The binary symmetric channel, assuming a channel noise modeled as additive white 
Gaussian noise (AWGN) channel, is completely described by the transition probability p. 
The majority of coded digital communication systems employ binary coding with hard- 
decision decoding, due to the simplicity of implementation offered by such an approach. 
Hard-decision decoders, or algebraic decoders, take advantage of the special algebraic 
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Figure 10.2 Transition probability diagram of binary symmetric channel. 
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FIGURE 10.3 Binary input £)-ary output discrete memoryless channel, (a) Receiver for binary 
phase-shift keying, (h) Transfer characteristic of multilevel quantizer, (c) Channel transition prob- 
ability diagram. Parts (b) and (c) are illustrated for eight levels of quantization. 



structure that is built into the design of channel codes to make the decoding relatively easy 
to perform. 

The use of hard decisions prior to decoding causes an irreversible loss of information 
in the receiver. To reduce this loss, soft-decision coding is used. This is achieved by in- 
cluding a multilevel quantizer at the demodulator output, as illustrated in Figure 10.3a 
for the case of binary PSK signals. The input-output characteristic of the quantizer is 
shown in Figure 10.36. The modulator has only the binary symbols 0 and 1 as inputs, but 
the demodulator output now has ah alphabet with Q symbols. Assuming the use of the 
quantizer as described in Figure 10.3fc, we have Q = 8. Such a channel is called a binary 
input Q-ary output discrete memoryless channel. The corresponding channel transition 
probability diagram is shown in Figure 10.3c. The form of this distribution, and conse- 
quently the decoder performance, depends on the location of the representation levels of 
the quantizer, which, in turn, depends on the signal level and noise variance. Accordingly, 
the demodulator must incorporate automatic gain control if an effective multilevel quan- 
tizer is to be realized. Moreover, the use of soft decisions complicates the implementation 
of the decoder. Nevertheless, soft-decision decoding offers significant improvement in per- 
formance over hard-decision decoding by taking a probabilistic rather than an algebraic 
approach. It is for this reason that soft-decision decoders are also referred to as probabi- 
listic decoders. 



■ Channel Coding Theorem Revisited 

In Chapter 9, we established the concept of channel capacity, which, for a discrete 
memoryless channel, represents the maximum amount of information transmitted per 
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channel use. The channel coding theorem states that if a discrete memoryless channel has 
capacity C and a source generates information at a rate less than C, then there exists a 
coding technique such that the output of the source may be transmitted over the channel 
with an arbitrarily low probability of symbol error. For the special case of a binary sym- 
metric channel, the theorem tells us that if the code rate r is less than the channel capacity 
C, then it is possible to find a code that achieves error-free transmission over the channel. 
Conversely, it is not possible to find such a code if the code rate r is greater than the 
channel capacity C. 

The channel coding theorem thus specifies the channel capacity C as a fundamental 
limit on the rate at which the transmission of reliable (error-free) messages can take place 
over a discrete memoryless channel. The issue that matters is not the signal-to-noise ratio, 
so long as it is large enough, but how the channel input is encoded. 

The most unsatisfactory feature of the channel coding theorem, however, is its non- 
constructive nature. The theorem asserts the existence of good codes but does not tell us 
how to find them. By good codes we mean families of channel codes that are capable of 
providing reliable transmission of information (i.e., at arbitrarily small probability of sym- 
bol error) over a noisy channel of interest at bit rates up to a maximum value less than 
the capacity of that channel. The error-control coding techniques described in this chapter 
provide different methods of designing good codes. 



■ Notation 

The codes described in this chapter are binary codes, for which the alphabet consists only 
of symbols 0 and 1 . In such a code, the encoding and decoding functions involve the binary 
arithmetic operations of modulo-2 addition and multiplication performed on code words 
in the code. 

Throughout this chapter, we use an ordinary plus sign (+) to denote modulo-2 ad- 
dition. The use of this terminology will not lead to confusion because the whole chapter 
relies on binary arithmetic. In so doing, we avoid the use of a special symbol ©, as we did 
in preceding chapters. Thus, according to the notation used in this chapter, the rules for 
modulo-2 addition are as follows: 

0 + 0 = 0 

1 + 0 = 1 

0 + 1 = 1 

1 + 1 = 0 

Because 1 + 1 = 0, it follows that 1 = -1. Hence, in binary arithmetic, subtraction is the 
same as addition. The rules for modulo-2 multiplication are as follows: 

0X0 = 0 
1X0 = 0 
0X1 = 0 
1X1 = 1 

Division is trivial in that we have 
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and division by 0 is not permitted. Modulo-2 addition is the EXCLUSIVE-OR operation 
in logic, and modulo-2 multiplication is the AND operation. 



10.3 Linear Block Codes 

A code is said to be linear if any two code words in the code can be added in modulo-2 
arithmetic to produce a third code word in the code. Consider then an («, k) linear block 
code, in which k bits of the n code bits are always identical to the message sequence to be 
transmitted. The n — k bits in the remaining portion are computed from the message bits 
in accordance with a prescribed encoding rule that determines the mathematical structure 
of the code. Accordingly, these n — k bits are referred to as generalized parity check bits 
or simply parity bits. Block codes in which the message bits are transmitted in unaltered 
form are called systematic codes. For applications requiring both error detection and error 
correction, the use of systematic block codes simplifies implementation of the decoder. 

Let m 0 , mi, ■ ■ ■ , mk-i constitute a block of k arbitrary message bits. Thus we have 
2 k distinct message blocks. Let this sequence of message bits be applied to a linear block 
encoder, producing an n-bit code word whose elements are denoted by c 0 , c x , . . . , c n ^ x . 
Let b 0 , fe„-fc-i denote the [n - k) parity bits in the code word. For the code to 

possess a systematic structure, a code word is divided into two parts, one of which is 
occupied by the message bits and the other by the parity bits. Clearly, we have the option 
of sending the message bits of a code word before the parity bits, or vice versa. The former 
option is illustrated in Figure 10.4, and its use is assumed in the sequel. 

According to the representation of Figure 10.4, the (n — k) left-most bits of a code 
word are identical to the corresponding parity bits, and the k right-most bits of the code 
word are identical to the corresponding message bits. We may therefore write 

Ci JK ; = o,i .•■■.« - * -i m) 

lm i+k -„ y t = n - k, n - k + 1, ...,»- 1 

The (n - k) parity bits are linear sums of the k message bits, as shown by the generalized 
relation 

bi - p 0i m 0 + pvtn-L + • • • + pk-uMk-i ( 10 - 2 ) 
where the coefficients are defined as follows: 

(1 ii b t depends on m f ^ 
0 otherwise 

The coefficients p ; , are chosen in such a way that the rows of the generator matrix are 
linearly independent and the parity equations are unique. 

The system of Equations (10.1) and (10.2) defines the mathematical structure of the 
(n, k) linear block code. This system of equations may be rewritten in a compact form 
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Figure 10.4 Structure of systematic code word. 
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using matrix notation. To proceed with this reformulation, we define the 1-by-k message 
vector, or information vector, m, the l-by-(« - k) parity vector b, and the 1-by-n code 
vector c as follows: 



m = [m 0 , 

b = [b 0 , £>!,..., £>„_*- 1] 
c = [c 0 , c 1; . . . , c„_,]_ 



(10.4) 
(10.5) 
(10.6) 



Note that all three vectors are row vectors. The use of row vectors is adopted in this chapter 
for the sake of being consistent with the notation commonly used in the coding literature. 
We may thus rewrite the set of simultaneous equations defining the parity bits in the 
compact matrix form: 

b = mP (10.7) 
where P is the k-by-(n - k) coefficient matrix defined by 



(10.8) 



where p # is 0 or 1, 

From the definitions given in Equations (10.4)-(10.6), we see that c may be expressed 
as a partitioned row vector in terms of the vectors m and b as follows: 

c = f> ' m] (10.9) 

Hence, substituting Equation (10.7) into Equation (10.9) and factoring out the common 
message vector m, we get 
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where I k is the k-by-k identity matrix: 



Ik = 



c = m[P:IJ 

'1 0 ••• 
0 1 • • • 

0 0 ••• 



(10.10) 



(10.11) 



Define the k-by-n generator matrix 

G=[P:IJ (10.12) 
The generator matrix G of Equation (10.12) is said to be in the canonical form in that its 
k rows are linearly independent; that is, it is not possible to express any row of the matrix 
G as a linear combination of the remaining rows. Using the definition of the generator 
matrix G, we may simplify Equation (10.10) as 



c = raG 



(10.13) 



The full set of code words, referred to simply as the code, is generated in accordance 
with Equation (10.13) by letting the message vector m range through the set of all 2 k 
binary ^-tuples (1-by-k vectors). Moreover, the sum of any two code words is another 
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code word. This basic property of linear block codes is called closure. To prove its validity, 
consider a pair of code vectors c, and c, corresponding to a pair of message vectors m, and 
m„ respectively. Using Equation (10.13) we may express the sum of c ; and c, as 

c, + c ; = mjG + m/G 
= (nv + m ; -)G 

The modulo-2 sum of m, and m ; represents a new message vector. Correspondingly, the 
modulo-2 sum of c, and c, represents a new code vector. 

There is another way of expressing the relationship between the message bits and 
parity-check bits of a linear block code. Let H denote an (n - k)-by-n matrix, defined as 

H = [I„_ t : P 7 ] (10.14) 

where P T is an (n - k)-by-k matrix, representing the transpose of the coefficient matrix P, 
and l„- k is the (n - k)-by-{n - k) identity matrix. Accordingly, we may perform the 
following multiplication of partitioned matrices: 

= P T + P T 

where we have used the fact that multiplication of a rectangular matrix by an identity 
matrix of compatible dimensions leaves the matrix unchanged. In modulo-2 arithmetic, 
we have P T + P r = 0, where 0 denotes an (n - k)-by-k null matrix (i.e., a matrix that has 
zeros for all of its elements). Hence, 

HG r = 0 (10-15) 

Equivalently, we have GH T = 0, where 0 is a new null matrix. Postmultiplying both sides 
of Equation (10.13) by H T , the transpose of H, and then using Equation (10.15), we get 

cH T = mGH T (1(Ug) 
= 0 

The matrix H is called the parity-check matrix of the code, and the set of equations spec- 
ified by Equation (10.16) are called parity-check equations. 

The generator equation (10.13) and the parity-check detector equation (10.16) are 
basic to the description and operation of a linear block code. These two equations are 
depicted in the form of block diagrams in Figure 10.5a and 10.5fc, respectively. 
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Figure 10.5 Block diagram representations of the generator equation (10.13) and the parity- 
check equation (10.16). 
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► Example 10.1 Repetition Codes 

Repetition codes represent the simplest type of linear block codes. In particular, a single mes- 
sage bit is encoded into a block of n identical bits, producing an (», 1) block code. Such a 
code allows provision for a variable amount of redundancy. There are only two code words 
in the code: an all-zero code word and an all-one code word. 

Consider, for example, the case of a repetition code with k = 1 and n = 5. In this case, 
we have four parity bits that are the same as the message bit. Hence, the identity matrix Ij. = 
1, and the coefficient matrix P consists of a l-by-4 vector that has 1 for all of its elements. 
Correspondingly, the generator matrix equals a row vector of all Is, as shown by 

g = [i i i i ; i] 

The transpose of the coefficient matrix P, namely, matrix P T , consists of a 4-by-l vector that 
has 1 for all of its elements. The identity matrix I„_ t consists of a 4-by-4 matrix. Hence, the 
parity-check matrix equals 



H = 
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Since the message vector consists of a single binary symbol, 0 or 1, it follows from Equation 
(10.13) that there are only two code words: 00000 and 11111 in the (5, 1) repetition code, 
as expected. Note also that HG T = 0, modulo-2, in accordance with Equation (10.15). -4 



a Syndrome: Definition and Properties 

The generator matrix G is used in the encoding operation at the transmitter. On the other 
hand, the parity-check matrix H is used in the decoding operation at the receiver. In the 
context of the latter operation, let r denote the l-by-« received vector that results from 
sending the code vector c over a noisy channel. We express the vector r as the sum of the 
original code vector c and a vector e, as shown by 

r = c + e (10.17) 

The vector e is called the error vector or error pattern. The rth element of e equals 0 if the 
corresponding element of r is the same as that of c. On the other hand, the rth element of 
e equals 1 if the corresponding element of r is different from that of c, in which case an 
error is said to have occurred in the rth location. That is, for % = 1, 2, . . . , n, we have 

1 if an error has occurred in the rth location 
0 otherwise (ltU8) 

The receiver has the task of decoding the code vector c from the received vector r. 
The algorithm commonly used to perform this decoding operation starts with the com- 
putation of a l-by-(« - k) vector called the error-syndrome vector or simply the syn- 
drome. 3 The importance of the syndrome lies in the fact that it depends only upon the 
error pattern. 

Given a 1-by-K received vector r, the corresponding syndrome is formally defined as 

s = rH r (10.19) 
Accordingly, the syndrome has the following important properties. 
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Property 1 

The syndrome depends only on the error pattern, and not on the transmitted code word. 

To prove this property, we first use Equations (10.17) and (10.19), and then Equation 
(10.16) to obtain 

s = (c + e)H T 
= cH T + eH r (10.20) 
= eH T 

Hence, the parity-check matrix H of a code permits us to compute the syndrome s, which 
depends only upon the error pattern e. 

Property 2 

All error patterns that differ by a code word have the same syndrome. 

For k message bits, there are 2* distinct code vectors denoted as c j5 i = 0, 1, . . . , 
2* — 1. Correspondingly, for any error pattern e, we define the 2 k distinct vectors e, as 

e, = e + c i; i = 0, 1, . . . , 2 h - 1 (10.21) 

The set of vectors (e„ i = 0, 1, . . . , 2* - 1) so defined is called a coset of the code. In 
other words, a coset has exactly 2 k elements that differ at most by a code vector. Thus, 
an (n, k) linear block code has 2"~ h possible cosets. In any event, multiplying both sides 
of Equation (10.21) by the matrix H T , we get 

e^eH^c^ 
— eH 

which is independent of the index i. Accordingly, we may state that each coset of the code 
is characterized by a unique syndrome. 

We may put Properties 1 and 2 in perspective by expanding Equation (10.20). Spe- 
cifically, with the matrix H having the systematic form given in Equation (10.14), where 
the matrix P is itself defined by Equation (10.8), we find from Equation (10.20) that the 
(« - k) elements of the syndrome s are linear combinations of the n elements of the error 
pattern e, as shown by 



s 0 = e 0 + e„- k p 00 + e„- k ^p w + • • • + e„-ipt-i,o 
$! = £?! + e„~ k p 01 + e„-t+ipn + • • • + e„-ipt-i,i 



(10.23) 



Sn-k-l - Zn-k-l + e,i-kPo,n-k--L 



+ e„ 



This set of (n - k) linear equations clearly shows that the syndrome contains information 
about the error pattern and may therefore be used for error detection. However, it should 
be noted that the set of equations is underdetermined in that we have more unknowns 
than equations. Accordingly, there is no unique solution for the error pattern. Rather, 
there are 2" error patterns that satisfy Equation (10.23) and therefore result in the same 
syndrome, in accordance with Property 2 and Equation (10.22). In particular, with 2" 
possible syndrome vectors, the information contained in the syndrome s about the error 
pattern e is not enough for the decoder to compute the exact value of the transmitted code 
vector. Nevertheless, knowledge of the syndrome s reduces the search for the true error 
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pattern e from 2" to 2" k possibilities. Given these possibilities, the decoder has the task 
of making the best selection from the cosets corresponding to s. 

a Minimum Distance Considerations 

Consider a pair of code vectors c x and c 2 that have the same number of elements. The 
Hamming distance d(c,, c 2 ) between such a pair of code vectors is defined as the number 
of locations in which their respective elements differ. 

The Hamming weight w(c) of a code vector c is defined as the number of nonzero 
elements in the code vector. Equivalently, we may state that the Hamming weight of a 
code vector is the distance between the code vector and the all-zero code vector. 

The minimum distance d mia of a linear block code is defined as the smallest Hamming 
distance between any pair of code vectors in the code. That is, the minimum distance is 
the same as the smallest Hamming weight of the difference between any pair of code 
vectors. From the closure property of linear block codes, the sum (or difference) of two 
code vectors is another code vector. Accordingly, we may state that the minimum distance 
of a linear block code is the smallest Hamming weight of the nonzero code vectors in the 
code. 

The minimum distance 4ni„ is related to the structure of the parity-check matrix H 
of the code in a fundamental way. From Equation (10.16) we know that a linear block 
code is defined by the set of all code vectors for which cH T = 0, where H T is the transpose 
of the parity-check matrix H. Let the matrix H be expressed in terms of its columns as 
follows: 

H = [h„ h 2 , . . . , h„] (10.24) 

Then, for a code vector c to satisfy the condition cH T = 0, the vector c must have Is in 
such positions that the corresponding rows of H T sum to the zero vector 0. However, by 
definition, the number of Is in a code vector is the Hamming weight of the code vector. 
Moreover, the smallest Hamming weight of the nonzero code vectors in a linear block 
code equals the minimum distance of the code. Hence, the minimum distance of a linear 
block code is defined by the minimum number of rows of the matrix H T whose sum is 
equal to the zero vector. 

The minimum distance of a linear block code, d min , is an important parameter of the 
code. Specifically, it determines the error-correcting capability of the code. Suppose an 
(n, k) linear block code is required to detect and correct all error patterns (over a binary 
symmetric channel), and whose Hamming weight is less than or equal to t. That is, if a 
code vector c t in the code is transmitted and the received vector is r = q + e, we require 
that the decoder output c = c,-, whenever the error pattern e has a Hamming weight 
w(e) s t. We assume that the 2 h code vectors in the code are transmitted with equal 
probability. The best strategy for the decoder then is to pick the code vector closest to the 
received vector r, that is, the one for which the Hamming distance d(c h r) is the smallest. 
With such a strategy, the decoder will be able to detect and correct all error patterns of 
Hamming weight w(e) s t, provided that the minimum distance of the code is equal to or 
greater than 2t + 1. We may demonstrate the validity of this requirement by adopting a 
geometric interpretation of the problem. In particular, the l-by-« code vectors and the 
1-by-n received vector are represented as points in an n-dimensional space. Suppose that 
we construct two spheres, each of radius f, around the points that represent code vectors 
c, and c ; -. Let these two spheres be disjoint, as depicted in Figure 10.6a. For this condition 
to be satisfied, we require that d(c t , c ? ) £ It + 1. If then the code vector c, is transmitted 
and the Hamming distance d(c h r) £ t, it is clear that the decoder will pick c, as it is the 
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Figure 10.6 (a) Hamming distance d(c { , 2t + 1. (b) Hamming distance d(c it c^) < 2t. 
The received vector is denoted by r. 

code vector closest to the received vector r. If, on the other hand, the Hamming distance 
did, c ; ) £ 2f, the two spheres around c, and c ; intersect, as depicted in Figure 10.6b. Here 
we see that if c, is transmitted, there exists a received vector r such that the Hamming 
distance d(Ci, r) ^ f, and yet r is as close to 9 as it is to c ; . Clearly, there is now the 
possibility of the decoder picking the vector c ; , which is wrong. We thus conclude that an 
(n, k) linear block code has the power to correct all error patterns of weight t or less if, 
and only if, 

d(c h Cj) ^ 2t + 1 for all c, and c ; - 

By definition, however, the smallest distance between any pair of code vectors in a code 
is the minimum distance of the code, d^ n . We may therefore state that an (n, k) linear 
block code of minimum distance d^ can correct up to t errors if, and only if, 

t =£ l{(d^ - 1)J (10.25) 

where L J denotes the largest integer less than or equal to the enclosed quantity. Equation 
(10.25) gives the error-correcting capability of a linear block code a quantitative meaning. 

b Syndrome Decoding 

We are now ready to describe a syndrome-based decoding scheme for linear block codes. 
Let d, c 2 , . . . , c 2 * denote the 2 k code vectors of an (n, k) linear block code. Let r denote 
the received vector, which may have one of 2" possible values. The receiver has the task 
of partitioning the 2" possible received vectors into 2 k disjoint subsets £2),, 26 2a . . . , 2> 2 * in 
such a way that the rth subset 9)< corresponds to code vector c f for 1 < ;' < 2 k . The received 
vector r is decoded into c f if it is in the rth subset. For the decoding to be correct, r must 
be in the subset that belongs to the code vector c, that was actually sent. 

The 2* subsets described herein constitute a standard array of the linear block code. 
To construct it, we may exploit the linear structure of the code by proceeding as follows: 

1. The 2 k code vectors are placed in a row with the all-zero code vector c a as the left- 
most element. 

2. An error pattern ^ is picked and placed under c u and a second row is formed by 
adding e 2 to each of the remaining code vectors in the first row; it is important that 
the error pattern chosen as the first element in a row not have previously appeared 
in the standard array. 

3. Step 2 is repeated until all the possible error patterns have been accounted for. 

Figure 10.7 illustrates the structure of the standard array so constructed. The 2 k columns 
of this array represent the disjoint subsets SS 1} £J 2 i • • • > 22 2 '- The 2"~ k rows of the array 
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Figure 10.7 Standard array for an (», fc) block code. 



represent the cosets of the code, and their first elements e 2 , . . . , e 2 «-t are called cost* 
leaders. 

For a given channel, the probability of decoding error is minimized when the most 
likely error patterns (i.e., those with the largest probability of occurrence) are chosen as 
the coset leaders. In the case of a binary symmetric channel, the smaller the Hamming 
weight of an error pattern the more likely it is to occur. Accordingly, the standard array 
should be constructed with each coset leader having the minimum Hamming weight in its 
coset. 

We may now describe a decoding procedure for a linear block code: 

1. For the received vector r, compute the syndrome s = rH T , 

2. Within the coset characterized by the syndrome s, identify the coset leader (i.e., the 
error pattern with the largest probability of occurrence); call it e 0 . 

3. Compute the code vector 

c = r + e 0 (10.26) 
as the decoded version of the received vector r. 
This procedure is called syndrome decoding. 

► Example 10.2 Hamming Codes 4 

Consider a family of («, k) linear block codes that have the following parameters: 
Block length: n = 2 m - 1 

Number of message bits: k = 2 m — m — 1 
Number of parity bits: n - k = m 

where m > 3. These are the so-called Hamming codes. 

Consider, for example, the (7, 4) Hamming code with n = 7 and k = 4, corresponding 
to m = 3. The generator matrix of the code must have a structure that conforms to Equation 
(10.12). The following matrix represents an appropriate generator matrix for the (7, 4) Ham- 
ming code: 



G = 



"1 
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0 


1 


0 0 


0" 


0 


1 


1 


0 


1 0 


0 


1 


1 


1 


0 


0 1 
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_1_ 


0 
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0 


0 0 


1. 
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Table 10.1 Code words of a (7, 4) Hamming code 



Message 
Word 


Lode word 


Weight of 
Code Word 


Message 
Word 


Code Word 


Weight of 
Code Word 


0 0 0 0 


0000000 


0 


10 0 0 


110 10 0 0 


3 


0 0 0 1 


1010001 


3 


10 0 1 


0 1110 0 1 


4 


0010 


1110010 


4 


10 10 


0011010 


3 


0 0 11 


0 10 0011 


• 3 


10 11 


1001011 


4 


0 10 0 


0 110 10 0 


3 


110 0 


10 1110 0 


4 


0 10 1 


1100101 


4 


110 1 


0 0 0 110 1 


3 


0 110 


10 0 0 110 


3 


1110 


0 10 1110 


4 


0 111 


0 0 10 111 


4 


1111 


1111111 


7 



The corresponding parity-check matrix is given by 





"l 


0 


0 


1 


0 


1 


1 


H = 


0 


1 


0 


1 


1 


1 


0 




0_ 


0 


J 


0 


1 


1 


1 



With k = 4, there are 2* = 16 distinct message words, which are listed in Table 
10.1. For a given message word, the corresponding code word is obtained by using Equa- 
tion (10.13). Thus, the application of this equation results in the 16 code words listed in 
Table 10.1. 

In Table 10.1, we have also listed the Hamming weights of the individual code words 
in the (7, 4) Hamming code. Since the smallest of the Hamming weights for the nonzero code 
words is 3, it follows that the minimum distance of the code is 3. Indeed, Hamming codes 
have the property that the minimum distance d min = 3, independent of the value assigned to 
the number of parity bits m. 

To illustrate the relation between the minimum distance d^ and the structure of the 
parity-check matrix H, consider the code word 0110100. In the matrix multiplication denned 
by Equation (10.16), the nonzero elements of this code word "sift" out the second, third, and 
fifth columns of the matrix H yielding 



0 




0 




0 




0 


1 


+ 


0 


+ 


1 




0 


0 




1 




1 




0 



We may perform similar calculations for the remaining 14 nonzero code words. We thus find 
that the smallest number of columns in H that sums to zero is 3, confirming the earlier state- 
ment that d^ = 3. 

An important property of Hamming codes is that they satisfy the condition of Equation 
(10.25) with the equality sign, assuming that t = 1. This means that Hamming codes ate 
single-error correcting binary perfect codes. 

Assuming single-error patterns, we may formulate the seven coset leaders listed in the 
right-hand column of Table 10.2. The corresponding 2 3 syndromes, listed in the left-hand 
column, are calculated in accordance with Equation (10.20), The zero syndrome signifies no 
transmission errors. 

Suppose, for example, the code vector [1110010] is sent, and the received vector is 
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I Table 10.2 Decoding 
* table for the (7,4) 
;j Hamming code defined 
t in Table 10.1 



Syndrome 


Error Pattern 


000 


0 0 0 0 0 0 0 


1 0 0 


10 0 0 0 0 0 


0 10 


0 10 0 0 0 0 


001 


0 0 10 0 0 0 


1 1 0 


0 0 0 10 0 0 


0 1 1 


0 0 0 0 10 0 


1 1 1 


0 0 0 0 0 10 


10 1 


0 0 0 0 0 0 1 



[1100010] with an error in the third bit. Using Equation (10.19), the syndrome is calculated 
to be 



[1100010] 



= [0 0 1] 



From Table 10.2 the corresponding coset leader (i.e., error pattern with the highest probability 
of occurrence) is found to be [0010000], indicating correctly that the third bit of the received 
vector is erroneous. Thus, adding this error pattern to the received vector, in accordance with 
Equation (10.26), yields the correct code vector actually sent. «< 



■ Dual Code 

Given a linear block code, we may define its dual as follows. Taking the transpose of both 
sides of Equation (10.15), we have 

GH T = 0 

where H T is the transpose of the parity-check matrix of the code, and 0 is a new zero 
matrix. This equation suggests that every (n, k) linear block code with generator matrix 
G and parity-check matrix H has a dual code with parameters (n,n — k), generator matrix 
H and parity-check matrix G. 



| 10.4 Cyclic Codes 



Cyclic codes form a subclass of linear block codes. Indeed, many of the important linear 
block codes discovered to date are either cyclic codes or closely related to cyclic codes. An 
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advantage of cyclic codes over most other types of codes is that they are easy to encode 
Furthermore, cyclic codes possess a well-defined mathematical structure, which has led t' 
the development of very efficient decoding schemes for them. ° 
A binary code is said to be a cyclic code if it exhibits two fundamental properties- 

1. Linearity property: The sum of any two code words in the code is also a code word 

2. Cyclic property: Any cyclic shift of a code word in the code is also a code word. 

Property 1 restates the fact that a cyclic code is a linear block code (i.e., it can be described 
as a parity-check code). To restate Property 2 in mathematical terms, let the «-tupl e 
(c 0 , c u . . . , c„_i) denote a code word of an (», k) linear block code. The code is a cyclic 
code if the «-tuples 

c 0 , . . . , C„- 2 ), 
( c «-2) • - - , C M _ 3 ), 

( c l> C 2 , . . . , C„_!, C 0 ) 

are all code words in the code. 

To develop the algebraic properties of cyclic codes, we use the elements c 0 , c u . . . 
c„_! of a code word to define the code polynomial ' 

c(X) = c 0 + Cl X + c 2 X 2 + ■■■ + c^X"- 1 (10.27) 

where X is an indeterminate. Naturally, for binary codes, the coefficients are Is and Os. 
Each power of X in the polynomial c(X) represents a one-bit shift in time. Hence, multi- 
plication of the polynomial c(X) by X may be viewed as a shift to the right. The key 
question is: How do we make such a shift cyclic} The answer to this question is addressed 
next. 

Let the code polynomial c(X) be multiplied by X', yielding 

X l c(X) = X'(c 0 + c 1 X + --- + c.^X"-'- 1 + c„^X"-' 

+ ■■■ + c^X"- 1 ) 
= c 0 X' + Cl X M + ■■■ + c^.X"- 1 + C „_,X" 

+ --- + C „_ 1 X— < 10 ' 28 ' 
= c„_ ( X" + • • • + c^X"*'- 1 + c 0 X< + Cl X i+1 

+ • • • + c^X"- 1 

where, in the last line, we have merely rearranged terms. Recognizing, for example, that 
c„~i + £„-,- = 0 in modulo-2 addition, we may manipulate the first i terms of Equation 
(10.28) as follows: 

X'c(X) = c „_,. + ■ ■ ■ + c^X''- 1 + c 0 X> + Cl X i+1 + • • • + c^X"- 1 

+ c,(X' + i) + • • - + ^x'-^x" + i) (10 " 29) 

Next, we introduce the following definitions: 

c w (X) = c„_,. + ■ • • + Cn-iX'- 1 + coX' + Cl X' +1 

+ - + (ia30) 

q(X) = c„-i + c„_i +1 X + • • • + c-jX'- 1 (10.31) 
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Accordingly, Equation (10.29) is reformulated in the compact form 

X'c(X) = q(X)(X" + 1) + c«(X) (10.32) 

The polynomial c (,) (X) is recognized as the code polynomial of the code word (c„- h . . . , 
Cn-i, c 0 , c 1} . . ., c„-i-i) obtained by applying i cyclic shifts to the code word (c 0 , c x , . . , , 
c n -i-i, c„_„ . . . , c„_i). Moreover, from Equation (10.32) we readily see that c m (X) is the 
remainder that results from dividing X'c(X) by (X" + 1). We may thus formally state the 
cyclic property in polynomial notation as follows: If c(X) is a code polynomial, then the 
polynomial 

c<"(X) = X'c(X) mod(X" + 1) (10.33) 

is also a code polynomial for any cyclic shift i; the term mod is the abbreviation for modulo. 
The special form of polynomial multiplication described in Equation (10.33) is referred to 
as multiplication modulo X" + 1. In effect, the multiplication is subject to the constraint 
X" = 1, the application of which restores the polynomial X'c(X) to order n — 1 for all 
/ < n. (Note that in modulo-2 arithmetic, X" + 1 has the same value as X" - 1.) 

a Generator Polynomial 

The polynomial X" + 1 and its factors play a major role in the generation of cyclic codes. 
Let g(X) be a polynomial of degree n - k that is a factor of X" + 1; as such, g(X) is the 
polynomial of least degree in the code. In general, g(X) may be expanded as follows: 

g(X) = 1+2 giX' + X"~ b (10.34) 
•-1 

where the coefficient g, is equal to 0 or 1. According to this expansion, the polynomial 
g{X) has two terms with coefficient 1 separated by n - k - 1 terms. The polynomial g(X) 
is called the generator polynomial of a cyclic code. A cyclic code is uniquely determined 
by the generator polynomial g(X) in that each code polynomial in the code can be ex- 
pressed in the form of a polynomial product as follows: 

c(X) = a(X)g(X) (10.35) 

where a(X) is a polynomial in X with degree k-1. The c(X) so formed satisfies the 
condition of Equation (10.33) since g(X) is a factor of X" + 1. 

Suppose we are given the generator polynomial g(X ) and the requirement is to encode 
the message sequence {m 0 , m u . . . , m k -i) into an (n, k) systematic cyclic code. That is, 
the message bits are transmitted in unaltered form, as shown by the following structure 
for a code word (see Figure 10.4): 

( b 0 , fr„-t-i , m 0 , m u . . . , m t -i) 

n — k parity bits k message bits 

Let the message polynomial be defined by 

m(X) = m 0 + mi X + ■ • - + m k „ 1 X b ~ 1 (10.36) 

and let 



b(X) = b 0 + b t X + 



+ K^X' 



(10.37) 
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According to Equation (10.1), we want the code polynomial to be in the form 

c(X) = b(X) + X"- k m{X) (10.38) 

Hence, the use of Equations (10.35) and (10.38) yields 

a(X)g(X) = b{X) + X n - k m(X) 

Equivalently, in light of modulo-2 addition, we may write 
X"- k m(X) _ b(X) 

-^xr^ a{X) + W) (1039 > 

Equation (10.39) states that the polynomial b{X) is the remainder left over after dividing 
X"- k m(X)byg(X). 

We may now summarize the steps involved in the encoding procedure for an («, k) 
cyclic code assured of a systematic structure. Specifically, we proceed as follows: 

1. Multiply the message polynomial m(X) by X n ~ k . 

2. Divide X"~ k m{X) by the generator polynomial g(X), obtaining the remainder b(X). 

3. Add b{X) to X"~ k m(X), obtaining the code polynomial c(X). 



m Parity-Check Polynomial 

An (n, k) cyclic code is uniquely specified by its generator polynomial g(X) of order 
(n — k). Such a code is also uniquely specified by another polynomial of degree k, which 
is called the parity-check polynomial, denned by 

4-1 

h(X) = 1 + 2 h,X' + X k (10.40) 

where the coefficients h t are 0 or 1. The parity-check polynomial h(X) has a form similar 
to the generator polynomial in that there are two terms with coefficient 1, but separated 
by k — 1 terms. 

The generator polynomial g(X) is equivalent to the generator matrix G as a descrip- 
tion of the code. Correspondingly, the parity-check polynomial, denoted by h{X), is an 
equivalent representation of the parity-check matrix H. We thus find that the matrix re- 
lation HG T = 0 presented in Equation (10.15) for linear block codes corresponds to the 
relationship 

g(X)h(X) mod(X" + 1) = 0 (10.41) 

Accordingly, we may state that the generator polynomial g(X) and the parity-check poly- 
nomial h(X) are factors of the polynomial X" + 1, as shown by 

g(X)h(X) = X" + 1 (10.42) 

This property provides the basis for selecting the generator or parity-check polynomial of 
a cyclic code. In particular, we may state that if g(X) is a polynomial of degree (n— k) 
and it is also a factor of X"+ 1, then g(X) is the generator polynomial of an (n, k) cyclic 
code. Equivalently, we may state that if h(X) is a polynomial of degree k and it is also a 
factor of X" + 1, then h(X) is the parity-check polynomial of an (n, k) cyclic code. 

A final comment is in order. Any factor of X" + 1 with degree (n — k), the number 
of parity bits, can be used as a generator polynomial. For large values of n, the polynomial 
X" + 1 may have many factors of degree n — k. Some of these polynomial factors generate 
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good cyclic codes, whereas some of them generate bad cyclic codes. The issue of how to 
select generator polynomials that produce good cyclic codes is very difficult to resolve. 
Indeed, coding theorists have expended much effort in the search for good cyclic codes. 

a Generator and Parity-Check Matrices 

Given the generator polynomial g(X) of an («, k) cyclic code, we may construct the gen- 
erator matrix G of the code by noting that the k polynomials g(X), Xg(X), . . . , X k ~ x g(X) 
span the code. Hence, the «-tuples corresponding to these polynomials may be used as 
rows of the k-by-n generator matrix G. 

However, the construction of the parity-check matrix H of the cyclic code from the 
parity-check polynomial h(X) requires special attention, as described here. Multiplying 
Equation (10.42) by a(x) and then using Equation (10.35), we obtain 

c(X)h{X) = a(X) + X"a{X) (10.43) 

The polynomials c(X) and h(X) are themselves defined by Equations (10.27) and (10.40), 
respectively, which means that their product on the left-hand side of Equation (10.43) 
contains terms with powers extending up to n + k — 1. On the other hand, the polynomial 
a(X) has degree k — 1 or less, the implication of which is that the powers of X 4 , X k+1 , . . . , 
X" -1 do not appear in the polynomial on the right-hand side of Equation (10.43). Thus, 
setting the coefficients of X k , X i_1 , . . . , X" -1 in the expansion of the product polynomial 
c(X)h(X) equal to zero, we obtain the following set of n — k equations: 

i+k 

2 Cih k+M = 0 for 0 <= / <= n - k - 1 (10.44) 

<"=; 

Comparing Equation ( 1 0,44) with the corresponding relation of Equation ( 1 0. 1 6), we may 
make the following important observation: The coefficients of the parity-check polynomial 
h{X) involved in the polynomial multiplication described in Equation (10.44) are arranged 
in reversed order with respect to the coefficients of the parity-check matrix H involved in 
forming the inner product of vectors described in Equation (10.16). This observation sug- 
gests that we define the reciprocal of the parity-check polynomial as follows: 

/ t-i 

x^x- 1 ) = x*[ 1 + 2 bp-' + X~ h 

k-i 

= i + 2 V«x ; + x k 

which is also a factor of X™ + 1. The n-tuples pertaining to the (n — k) polynomials 
X^X" 1 ), X k+1 h(X-\ X^HX- 1 ) may now be used in rows of the (« - k)-hy-n 
parity-check matrix H. 

In general, the generator matrix G and the parity-check matrix H constructed in the 
manner described here are not in their systematic forms. They can be put into their sys- 
tematic forms by performing simple operations on their respective rows, as illustrated in 
Example 10.3. 

a Encoder for Cyclic Codes 

Earlier we showed that the encoding procedure for an («, k) cyclic code in systematic form 
involves three steps: (1) multiplication of the message polynomial m(X) by X"~ k , (2) di- 
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Figure 10.8 Encoder for an («, k) cyclic code. 



vision of X n ~ k m(X) by the generator polynomial g(X) to obtain the remainder b(X), and 
(3) addition of b(X) to X"~ k m(X) to form the desired code polynomial. These three steps 
can be implemented by means of the encoder shown in Figure 10.8, consisting of a linear 
feedback shift register with (« — k) stages. 

The boxes in Figure 10.8 represent flip-flops, or unit-delay elements. The flip-flop is 
a device that resides in one of two possible states denoted by 0 and 1. An external clock 
(not shown in Figure 10.8) controls the operation of all the flip-flops. Every time the clock 
ticks, the contents of the flip-flops (initially set to the state 0) are shifted out in the direction 
of the arrows. In addition to the flip-flops, the encoder of Figure 10.8 includes a second 
set of logic elements, namely, adders, which compute the modulo-2 sums of their respective 
inputs. Finally, the multipliers multiply their respective inputs by the associated coeffi- 
cients. In particular, if the coefficient g, = 1, the multiplier is just a direct "connection." 
If, on the other hand, the coefficient g, = 0, the multiplier is "no connection." 

The operation of the encoder shown in Figure 10.8 proceeds as follows: 

1. The gate is switched on. Hence, the k message bits are shifted into the channel. As 
soon as the k message bits have entered the shift register, the resulting (« — k) bits 
in the register form the parity bits [recall that the parity bits are the same as the 
coefficients of the remainder b(X)]. 

2. The gate is switched off, thereby breaking the feedback connections. 

3. The contents of the shift register are read out into the channel. 

a Calculation of the Syndrome 

Suppose the code word {c a , c u . . . , c„-y\ is transmitted over a noisy channel, resulting in 
the received word (r 0 , r 1} . . ., r„-i). From Section 10.3, we recall that the first step in the 
decoding of a linear block code is to calculate the syndrome for the received word. If the 
syndrome is zero, there are no transmission errors in the received word. If, on the other 
hand, the syndrome is nonzero, the received word contains transmission errors that require 
correction. 

In the case of a cyclic code in systematic form, the syndrome can be calculated easily. 
Let the received word be represented by a polynomial of degree n — 1 or less, as shown 
by 



r(X) = r 0 + r,X +.-•■ + r„_ a X" 



(10.46) 
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Let q(X) denote the quotient and s(X) denote the remainder, which are the results of 
dividing r(X) by the generator polynomial g(X). We may therefore express r(X) as follows: 

r(X) = q(X)g(X) + s(X) (10.47) 

The remainder s(X) is a polynomial of degree n — k — 1 or less, which is the result of 
interest. It is called the syndrome polynomial because its coefficients make up the (n — k)- 
by-1 syndrome s. 

Figure 10.9 shows a syndrome calculator that is identical to the encoder of Figure 
10.8 except for the fact that the received bits are fed into the {n - k) stages of the feedback 
shift register from the left. As soon as all the received bits have been shifted into the shift 
register, its contents define the syndrome s. 

The syndrome polynomial s(X) has the following useful properties that follow from 
the definition given in Equation (10.47). 

1. The syndrome of a received word polynomial is also the syndrome of the corresponding 
error polynomial. 

Given that a cyclic code with polynomial c(X) is sent over a noisy channel, the received 
word polynomial is defined by 

r(X) = c(X) + e(X) (10.48) 
where e(X) is the error polynomial. Equivalently, we may write 

e(X) = r(X) + c(X) (10.49) 

Hence, substituting Equations (10.35) and (10.47) into (10.49), we get 

e(X) = u(X)g(X) + s(X) (10.50) 

where the quotient is u(X) = a(X) + q(X). Equation (10.50) shows that s(X) is also the 
syndrome of the error polynomial e(X). The implication of this property is that when the 
syndrome polynomial s(X) is nonzero, the presence of transmission errors in the received 
word is detected. 

2. Let s(X) be the syndrome of a received word polynomial r(X). Then, the syndrome of 
Xr(X), a cyclic shift ofr(X), is Xs(X). 

Applying a cyclic shift to both sides of Equation (10.47), we get 

Xr(X) = Xq(X)g(X) + Xs(X) (10.51) 




Flip-flop Modulo-2 
adder 



FIGURE 10.9 Syndrome calculator for («, k) cyclic code. 
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from which we readily see that Xs(X) is the remainder of the division of Xr(X) by g(X) 
Hence, the syndrome of Xr(X) is Xs(X) as stated. We may generalize this result by stating 
that if s(X) is the syndrome of r(X), then X's(X) is the syndrome of X'r(X). 

3. The syndrome polynomial s(X) is identical to the error polynomial e{X), assuming that 
the errors are confined to the (n — k) parity-check bits of the received word polynomial 
r(X). 

The assumption made here is another way of saying that the degree of the error polynomial 
e(X) is less than or equal to (n — k — 1). Since the generator polynomial g(X) is of degree 
(n — k), by definition, it follows that Equation (10.50) can only be satisfied if the quotient 
«(X) is zero. In other words, the error polynomial e(X) and the syndrome polynomial s(X) 
aie one and the same. The implication of Property 3 is that, under the aforementioned 
conditions, error correction can be accomplished simply by adding the syndrome poly- 
nomial s(X) to the received word polynomial r(X). 

► Example 10.3 Hamming Codes Revisited 

To illustrate the issues relating to the polynomial representation of cyclic codes, we consider 
the generation of a (7, 4) cyclic code. With the block length n = 7, we start by factorizing 
X 7 + 1 into three irreducible polynomials: 

X 7 + 1 = (1 + X)(l + X 2 + X 3 )(l + X + X 3 ) 

By an "irreducible polynomial" we mean a polynomial that cannot be factored using only 
polynomials with coefficients from the binary field. An irreducible polynomial of degree 
m is said to be primitive if the smallest positive integer n for which the polynomial divides 
X" + 1 is n = 2 m — 1. For the example at hand, the two polynomials (1 + X 2 + X 3 ) and 
(1 + X + X 3 ) are primitive. Let us take 

g(X) = 1 + X + X 3 

as the generator polynomial, whose degree equals the number of parity bits. This means that 
the parity-check polynomial is given by 

h(X) = (1 + X)(l +X 2 + X 3 ) 
= 1 + X + X 2 + X 4 

whose degree equals the number of message bits k = 4. 

Next, we illustrate the procedure for the construction of a code word by using this 
generator polynomial to encode the message sequence 1001. The corresponding message poly- 
nomial is given by 

m(X) = 1 + X 3 
Hence, multiplying m(X) by X"~ k = X 3 , we get 

X"- k m(X) = X 3 + X 6 

The second step is to divide X"~ h m(X) by g(X), the details of which (for the example at hand) 
are given below: 

X 3 + X 

X 3 + X + ljx 5 +X 1 

X 6 + X" + X 3 

X 4 

X" + X 2 + X 

X 2 + X 
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Note that in this long division we have treated subtraction the same as addition, since we are 
operating in modulo-2 arithmetic. We may thus write 

X 3 + * 6 = v + V3 + X + X 2 
1+X + X 3 1+X + X 3 

That is, the quotient a(X) and remainder b{X) are as follows, respectively: 

a(X) = X + X 3 

b(X) = X + X 2 

Hence, from Equation (10.38) we find that the desired code polynomial is 

c{X) = b{X) + X"- h m{X) 

= X + X 2 + X 3 + X 6 

The code word is therefore 0111001. The four right-most bits, 1001, are the specified message 
bits. The three left-most bits, 011, are the parity-check bits. The code word thus generated is 
exactly the same as the corresponding one shown in Table 10.1 for a (7, 4) Hamming code. 

We may generalize this result by stating that any cyclic code generated by a primitive 
polynomial is a Hamming code of minimum distance 3. 

We next show that the generator polynomial g(X) and the parity-check polynomial 
h(X) uniquely specify the generator matrix G and the parity-check matrix H, respectively. 

To construct the 4-by-7 generator matrix G, we start with four polynomials represented 
by g(X) and three cyclic-shifted versions of it, as shown by 

g(X) = 1 + X + X 3 
Xg(X) = X + X 2 + X 4 
X 2 g{X) = X 2 + X 3 + X 5 
X 3 g(X) = X 3 + X 4 + X s 

The polynomials g(X), Xg(X), X 2 g(X), and X 3 g(X) represent code polynomials in the (7, 4) 
Hamming code. If the coefficients of these polynomials are used as the elements of the rows 
of a 4-by-7 matrix, we get the following generator matrix: 

"l 1 0 1 0 0 0" 

0 110 10 0 

G' = 

0 0 110 10 

_0 0 0 1 1 0 1_ 

Clearly, the generator matrix G' so constructed is not in systematic form. We can put it into 
a systematic form by adding the first row to the third row, and adding the sum of the first 
two rows to the fourth row. These manipulations result in the desired generator matrix: 

"l 1 0 1 0 0 0" 

0 110 10 0 

G = 

1110 0 10 

.1 0 1 0 0 0 1. 

which is exactly the same as that in Example 10.2. 

We next show how to construct the 3-by-7 parity-check matrix H from the parity-check 
polynomial h(X). To do this, we first take the reciprocal of h{X), namely, X 4 /j(X _1 ). For the 
problem at hand, we form three polynomials represented by X 4 h(X~ 1 ) and two shifted ver- 
sions of it, as shown by 

xmx- 1 ) = 1 + X 2 + X 3 + X 4 
X'hiX- 1 ) = X + X 3 + X 4 + X 5 
X^fX" 1 ) = X 2 + X 4 + X 5 + X 6 
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Figure 10.10 Encoder for the (7, 4) cyclic code generated by g(X) = 1 + X + X 3 , 



Using the coefficients of these three polynomials as the elements of the rows of the 3-by-7 
parity-check matrix, we get 



H' = 



Here again we see that the matrix H' is not in systematic form. To put it into a systematic 
form, we add the third row to the first row to obtain 

10 0 10 11 
0 10 1110 
0 0 10 111 

which is exactly the same as that of Example 10.2. 

Figure 10.10 shows the encoder for the (7, 4) cyclic Hamming code generated by the 
polynomial g(X) = 1 + X + X'. To illustrate the operation of this encoder, consider the 
message sequence (1001). The contents of the shift register are modified by the incoming 
message bits as in Table 10.3. After four shifts, the contents of the shift register, and therefore 
the parity bits, are (Oil). Accordingly, appending these parity bits to the message bits (1001 ), 
we get the code word (0111001); this result is exactly the same as that determined earlier in 
the example. 

Figure 10.11 shows the corresponding syndrome calculator for the (7, 4) Hamming 
code. Let the transmitted code word be (0111001) and the received word be (0110001); that 
is, the middle bit is in error. As the received bits are fed into the shift register, initially set to 
zero, its contents are modified as in Table 10.4. At the end of the seventh shift, the syndrome 
is identified from the contents of the shift register as 110. Since the syndrome is nonzero, the 
received word is in error. Moreover, from Table 10.2, we see that the error pattern corre- 
sponding to this syndrome is 0001000. This indicates that the error is in the middle bit of the 
received word, which is indeed the case. ^ 



Table 10.3 Contents of the 
shift register in the encoder 
of Figure 10.10 for message 
sequence (1001) 



Shift 


Input 


Register Contents 






0 0 0 (initial state) 


1 


1 


1 1 0 


2 


0 


0 1 1 


3 


0 


111 


4 


1 


0 1 1 
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I cyclic code generated by the polynomial 



Example 10.4 Maximal-Length Codes 

For any positive integer m > 3, there exists a maximal-length code with the following 
parameters: 

Block length: n - 2"' I 

Number of message bits: k = m 
Minimum distance: d min = 2 m " 1 

Maximal-length codes are generated by polynomials of the form 

* (X) = ; w (10 - 52) 

where h(X) is any primitive polynomial of degree m. Earlier we stated that any cyclic code 
generated by a primitive polynomial is a Hamming code of minimum distance 3 (see Example 
10.3). It follows therefore that maximal-length codes are the dual of Hamming codes. 

The polynomial h(X) defines the feedback connections of the encoder. The generator 
polynomial g(X) defines one period of the maximal-length code, assuming that the encoder is 
in the initial state 00 . . . 01. To illustrate these points, consider the example of a (7, 3) 
maximal-length code, which is the dual of the (7, 4) Hamming code described in Example 
10.3. Thus, choosing 

b(X) = 1 + X + X 3 
we find that the generator polynomial of the (7, 3) maximal-length code is 
g(X) = 1 + X + X 1 + X 4 



' TABLE 10.4 Contents of the syndrome 
; calculator in Figure 10.1 1 for the 
. received word 0110001 



Shift 


Input Bit 


Contents of Shift Register 






0 0 0 (initial state) 


1 


1 


1 0 0 


2 


0 


0 10 


3 


0 


00 1 


4 


0 


110 


5 


1 


111 


6 


1 


00 1 


7 


0 


110 
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Figure 10.12 Encoder for the (7, 3) maximal-length code; the initial state of the encoder is 
shown in the figure. 



Figure 10.12 shows the encoder for the (7, 3) maximal-length code, the feedback connections 
of which are exactly the same as those shown in Figure 8.2 in Chapter 8. The period of the 
code is n = 7. Thus, assuming that the encoder is in the initial state 001, as indicated in Figure 
10.12, we find the output sequence is described by 

100 1110100 

initial g(X) = 1 + X + X 2 + X 4 
state 

This result may be readily validated by cycling through the encoder of Figure 10.12. 
Note that if we were to choose the other primitive polynomial 

h(X) = 1 + X 2 + X 3 

for the (7, 3) maximal-length code, we would simply get the "image" of the code described 
above, and the output sequence would be "reversed" in time. *i 



■ Other Cyclic Codes 

We conclude the discussion of cyclic codes by presenting the characteristics of three other 
important classes of cyclic codes. 

Cyclic Redundancy Check Codes 

Cyclic codes are extremely well-suited for error detection. We make this statement 
for two reasons. First, they can be designed to detect many combinations of likely errors. 
Second, the implementation of both encoding and error- detecting circuits is practical. It is 
for these reasons that many of the error-detecting codes used in practice are of the cyclic- 
code type, A cyclic code used for error-detection is referred to as cyclic redundancy check 
(CRC) code. 

We define an error burst of length B in an n-bit received word as a contiguous 
sequence of B bits in which the first and last bits or any number of intermediate bits are 
received in error. Binary (n, k) CRC codes are capable of detecting the following error 
patterns: 

1. All error bursts of length n - k or less. 

2. A fraction of error bursts of length equal to n - k + 1; the fraction equals 
1 - 2~<-"- k -' ) . 

3. A fraction of error bursts of length greater than n - k + 1; the fraction equals 

1 _ 2-(»- fe - I > 

4. All combinations of d min - 1 (or fewer) errors. 

5. All error patterns with an odd number of errors if the generator polynomial g(X) 
for the code has an even number of nonzero coefficients. 
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| Table 10.5 


CKC codes 




Code 


ijCftBTCltOT rOlytlOtntCll} Q[J\J 


n — k 


CRC-12 code 


1 + X + X 2 + X 3 + X' 1 + X 12 


12 


CRC-16 code (USA) 


1 + X 2 + X 15 + X" 


16 


CRC-ITU code 


1 + X 5 + X 12 + X 16 


16 



Table 10.5 presents the generator polynomials of three CRC codes that have become 
international standards. All three codes contain 1 + X as a prime factor. The CRC-12 
code is used for 6-bit characters, and the other two codes are used for 8-bit characters. 
CRC codes provide a powerful method of error detection for use in automatic-repeat 
request (ARQ) strategies discussed in Section 10.1, and digital subscriber lines discussed 
in Chapter 4. 

Bose—Chaudhuri—Hocquenghem (BCH) Codes 5 

One of the most important and powerful classes of linear-block codes are BCH codes, 
which are cyclic codes with a wide variety of parameters. The most common binary BCH 
codes, known as primitive BCH codes, are characterized for any positive integers m (equal 
to or greater than 3) and t [less than (2 m — l)/2] by the following parameters: 

Block length: n = 2 m - 1 

Number of message bits: k a n — mt 

Minimum distance: d min a 2t + 1 

Each BCH code is a t-error correcting code in that it can detect and correct up to t random 
errors per code word. The Hamming single-error correcting codes can be described as 
BCH codes. The BCH codes offer flexibility in the choice of code parameters, namely, 
block length and code rate. Furthermore, for block lengths of a few hundred bits or less, 
the BCH codes are among the best known codes of the same block length and code rate. 

A detailed treatment of the construction of BCH codes is beyond the scope of 
our present discussion. To provide a feel for their capability, we present in Table 10.6, the 
code parameters and generator polynomials for binary block BCH codes of length up to 
2 s — 1. For example, suppose we wish to construct the generator polynomial for (15, 7) 

| Table 10.6 Binary BCH codes of length up to 2 s — 1 



n k t Generator Polynomial 



7 


4 


1 












1 


011 


15 


11 


1 












10 


011 


15 


7 


2 










111 


010 


001 


15 


5 


3 








10 


100 


110 


111 


31 


26 


1 












100 


101 


31 


21 


2 








11 


101 


101 


001 


31 


16 


3 




1 


000 


111 


110 


101 


111 


31 


11 


5 


101 


100 


010 


011 


011 


010 


101 


31 


6 


7 


11 001 Oil 


Oil 


110 


101 


000 


100 


111 



Notation; n = block length 

k = number of message bits 

f = maximum number of detectable errors 
The high-order coefficients of the generator polynomial g{X) are at the left. 
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BCH code. From Table 10.6 we have (111 010 001) for the coefficients of the generator 
polynomial; hence, we write 

g{X) = X s + X 7 + X 6 + X 4 + 1 

Reed— Solomon Codes 

The Reed-Solomon codes ate an important subclass of nonbinary BCH codes; they 
are often abbreviated as RS codes. The encoder for an RS code differs from a binary 
encoder in that it operates on multiple bits rather than individual bits. Specifically, an RS 
(«, k) code is used to encode m-bit symbols into blocks consisting of « = 2™ - 1 symbols, 
that is, m{2 m - 1) bits, where m > 1. Thus, the encoding algorithm expands a block of k 
symbols to n symbols by adding n - k redundant symbols. When m is an integer power 
of two, the m-bit symbols are called bytes. A popular value of m is 8; indeed, 8-bit RS 
codes are extremely powerful. 

A J-error-correcting RS code has the following parameters: 

Block length: n = 2™ - 1 symbols 

Message size: k symbols 

Parity-check size: n - k = 2t symbols 

Minimum distance: d min = It + 1 symbols 

The block length of the RS code is one less than the size of a code symbol, and the minimum 
distance is one greater than the number of parity-check symbols. The RS codes make highly 
efficient use of redundancy, and block lengths and symbol sizes can be adjusted readily to 
accommodate a wide range of message sizes. Moreover, the RS codes provide a wide range 
of code rates that can be chosen to optimize performance. Finally, efficient decoding tech- 
niques are available for use with RS codes, which is one more reason for their wide ap- 
plication (e.g., compact disc digital audio systems). 
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In block coding, the encoder accepts a k-bit message block and generates an «-bit code 
word. Thus, code words are produced on a block-by-block basis. Clearly, provision must 
be made in the encoder to buffer an entire message block before generating the associated 
code word. There are applications, however, where the message bits come in serially rather 
than in large blocks, in which case the use of a buffer may be undesirable. In such situa- 
tions, the use of convolutional coding may be the preferred method. A convolutional coder 
generates redundant bits by using modulo-2 convolutions, hence the name. 

The encoder of a binary convolutional code with rate 1/n, measured in bits per 
symbol, may be viewed as a finite-state machine that consists of an M-stage shift register 
with prescribed connections to n modulo-2 adders, and a multiplexer that serializes the 
outputs of the adders. An L-bit message sequence produces a coded output sequence of 
length «(L + M) bits. The code rate is therefore given by 

r = — bits/symbol (10.53) 

n(L + M) 1 

Typically, we have L » M. Hence, the code rate simplifies to ? 



r — - bits/symbol 
n 



(10.54) 
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The constraint length of a convolutional code, expressed in terms of message bits, is defined 
as the number of shifts over which a single message bit can influence the encoder output. 
In an encoder with an M-stage shift register, the memory of the encoder equals M message 
bits, and K = M + 1 shifts are required for a message bit to enter the shift register and 
finally come out. Hence, the constraint length of the encoder is K. 

Figure 10.13a shows a convolutional encoder with n = 2 and K = 3. Hence, the 
code rate of this encoder is 1/2. The encoder of Figure 10.13a operates on the incoming 
message sequence, one bit at a time. 

We may generate a binary convolutional code with rate kin by using k separate shift 
registers with prescribed connections to n modulo-2 adders, an input multiplexer and 
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Input 
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Figure 10.13 (a) Constraint Iength-3, rate-5 convolutional encoder, (b) Constraint length-2, 
rate-l convolutional encoder. 



656 Chapter 10 a Error-Control Coding 



an output multiplexer. An example of such an encoder is shown in Figure 10.13fc, 
where k = 2, n = 3, and the two shift registers have K = 2 each. The code rate is 2/3. In 
this second example, the encoder processes the incoming message sequence two bits at a 
time. 

The convolutional codes generated by the encoders of Figure 10.13 are nonsystematic 
codes. Unlike block coding, the use of nonsystematic codes is ordinarily preferred over 
systematic codes in convolutional coding. 

Each path connecting the output to the input of a convolutional encoder may be 
characterized in terms of its impulse response, defined as the response of that path to a 
symbol 1 applied to its input, with each flip-flop in the encoder set initially in the zero 
state. Equivalent^, we may characterize each path in terms of a generator polynomial, 
defined as the unit-delay transform of the impulse response. To be specific, let the generator 
sequence (gf, gf, g<i\ . . . , g [ Si) denote the impulse response of the rth path, where the 
coefficients gf, g ( {\ gf, ■ ■ ■ , g { Si equal 0 or 1. Correspondingly, the generator polynomial 
of the fth path is defined by 

gW(D) = gf + gfD + gfD 2 + ■■■+ g { Sfi M (10.55) 

where D denotes the unit-delay variable. The complete convolutional encoder is described 
by the set of generator polynomials {g m {D), g m (D), g M (D)). Traditionally, different 
variables are used for the description of convolutional and cyclic codes, with D being 
commonly used for convolutional codes and X for cyclic codes. 



► Example 10.5 

Consider the convolutional encoder of Figure 10.13a, which has two paths numbered 1 and 
2 for convenience of reference. The impulse response of path 1 (i.e., upper path) is (1, 1, 1). 
Hence, the corresponding generator polynomial is given by 

gU>(D) = 1 + D + D 2 

The impulse response of path 2 (i.e., lower path) is (1, 0, 1). Hence, the corresponding gen- 
erator polynomial is given by 

g (2 >(D) = 1 + D 2 

For the message sequence (10011), say, we have the polynomial representation 
m{D) = 1 + D 3 + D* 

As with Fourier transformation, convolution in the time domain is transformed into multi- 
plication in the D-domain. Hence, the output polynomial of path 1 is given by 

c m (D) = gW(D)m(D) 

= (1 + D + D 2 )(l + D 3 + D 4 ) 
= 1 + D + D 2 + D 3 + D 6 

From this we immediately deduce that the output sequence of path 1 is (1111001). Similarly, 
the output polynomial of path 2 is given by 

C W(D) = g< 2) (D)m(D) 

= (1 + D 2 )(l + D 3 + D 4 ) 

= 1 + D 2 + D 3 + D A + D 1 + D 6 
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The output sequence of path 2 is therefore (1011111). Finally, multiplexing the two output 
sequences of paths 1 and 2, we get the encoded sequence 

c = (11, 10, 11, 11, 01, 01, 11) 

Note that the message sequence of length L = 5 bits produces an encoded sequence of length 
n{L + K — 1) = 14 bits. Note also that for the shift register to be restored to its zero initial 
state, a terminating sequence of K — 1 = 2 zeros is appended to the last input bit of the 
message sequence. The terminating sequence of K — 1 zeros is called the tail of the message. 



m Code Tree, Trellis, and State Diagram 

Traditionally, the structural properties of a convolutional encoder are portrayed in graph- 
ical form by using any one of three equivalent diagrams: code tree, trellis, and state dia- 
gram. We will use the convolutional encoder of Figure 10.13a as a running example to 
illustrate the insights that each one of these three diagrams can provide. 

We begin the discussion with the code tree of Figure 10.14. Each branch of the tree 
represents an input symbol, with the corresponding pair of output binary symbols indi- 
cated on the branch. The convention used to distinguish the input binary symbols 0 and 
1 is as follows. An input 0 specifies the upper branch of a bifurcation, whereas input 1 
specifies the lower branch. A specific path in the tree is traced from left to right in accor- 
dance with the input (message) sequence. The corresponding coded symbols on the 
branches of that path constitute the input (message) sequence. Consider, for example, the 
message sequence (10011) applied to the input of the encoder of Figure 10.13a. Following 
the procedure just described, we find that the corresponding encoded sequence is 
(11, 10, 11,11, 01), which agrees with the first 5 pairs of bits in the encoded sequence {cj 
derived in Example 10.5. 

From the diagram of Figure 10.14, we observe that the tree becomes repetitive after 
the first three branches. Indeed, beyond the third branch, the two nodes labeled a are 
identical, and so are all the other node pairs that are identically labeled. We may establish 
this repetitive property of the tree by examining the associated encoder of Figure 10.13a. 
The encoder has memory M = K - 1 = 2 message bits. Hence, when the third message 
bit enters the encoder, the first message bit is shifted out of the register. Consequently, 
after the third branch, the message sequences (100 m 3 m 4 . . .) and (000 m 3 m 4 , . .) generate 
the same code symbols, and the pair of nodes labeled a may be joined together. The same 
reasoning applies to other nodes. Accordingly, we may collapse the code tree of Figure 

10.14 into the new form shown in Figure 10.15, which is called a trellis." It is so called 
since a trellis is a treelike structure with remerging branches. The convention used in Figure 

10.15 to distinguish between input symbols 0 and 1 is as follows. A code branch produced 
by an input 0 is drawn as a solid line, whereas a code branch produced by an input 1 is 
drawn as a dashed line. As before, each input (message) sequence corresponds to a specific 
path through the trellis. For example, we readily see from Figure 10.15 that the message 
sequence (10011) produces the encoded output sequence (11, 10, 11,11, 01), which agrees 
with our previous result. 

A trellis is more instructive than a tree in that it brings out explicitly the fact that 
the associated convolutional encoder is a finite-state machine. We define the state of a 
convolutional encoder of rate lln as the {K - 1) message bits stored in the encoder's shift 
register. At time /, the portion of the message sequence containing the most recent K bits 
is written as (mj- K -\, . . . , m t ), where ntj is the current bit. The (K — l)-bit state of 
the encoder at time / is therefore written simply as (m^-i, . . . , mf- K+ z, m^ K+1 ). In the 
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Figure 10.14 Code tree for the convolutional encoder of Figure 10.13a. 



case of the simple convolutional encoder of Figure 10.13*2 we have (K — 1) = 2. Hence, 
the state of this encoder can assume any one of four possible values, as described in Table 
10.7. The trellis contains (L + K) levels, where L is the length of the incoming message 
sequence, and K is the constraint length of the code. The levels of the trellis are labeled as 
/' = 0, 1, . . . , L + K - 1 in Figure 10.15 for K - 3. Level / is also referred to as depth /; 
both terms are used interchangeably. The first (K- 1) levels correspond to the encoder's 
departure from the initial state a, and the last (K - 1) levels correspond to the encoders 
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Level/=0 1 2 3 4 5 L-1 L L+l L+2_ 

Figure 10.15 Trellis for the c (involutional encoder of Figure 10.13a. 

return to the state a. Clearly, not all the states can be reached in these two portions of the 
trellis. However, in the central portion of the trellis, for which the level / lies in the range 
K — 1 £ /' £ L, all the states of the encoder are reachable. Note also that the central 
portion of the trellis exhibits a fixed periodic structure. 

Consider next a portion of the trellis corresponding to times / and /' + 1. We assume 
that / 2: 2 for the example at hand, so that it is possible for the current state of the encoder 
to be a, b, c, or d. For convenience of presentation, we have reproduced this portion of 
the trellis in Figure 10. 16a. The left nodes represent the four possible current states of the 
encoder, whereas the right nodes represent the next states. Clearly, we may coalesce the 
left and right nodes. By so doing, we obtain the state diagram of the encoder, shown in 
Figure 10.166. The nodes of the figure represent the four possible states of the encoder, 
with each node having two incoming branches and two outgoing branches. A transition 
from one state to another in response to input 0 is represented by a solid branch, whereas 
a transition in response to input 1 is represented by a dashed branch. The binary label on 
each branch represents the encoder's output as it moves from one state to another. Suppose, 
for example, the current state of the encoder is (01), which is represented by node c. The 
application of input 1 to the encoder of Figure 10.13a results in the state (10) and the 
encoded output (00). Accordingly, with the help of this state diagram, we may readily 
determine the output of the encoder of Figure 10.13a for any incoming message sequence. 
We simply start at state a, the all-zero initial state, and walk through the state diagram in 
accordance with the message sequence. We follow a solid branch if the input is a 0 and a 
dashed branch if it is a 1. As each branch is traversed, we output the corresponding binary 
label on the branch. Consider, for example, the message sequence (10011). For this input 
we follow the path abcabd, and therefore output the sequence (11, 10, 11, 11, 01), which 



Table 10.7 State table 
for the convolutional 
encoder of Figure 10.13a 



State 


Binary Description 


a 


00 


b 


10 


c 


01 


d 


11 
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agrees exactly with our previous result. Thus, the input-output relation of a convolutional 
encoder is also completely described by its state diagram. 



10.6 Maximum Like lihood 
Decoding of Convolutional Codes 

Now that we understand the operation of a convolutional encoder, the next issue to be 
considered is the decoding of a convolutional code. In this section we first describe the 
underlying theory of maximum likelihood decoding, and then present an efficient algo- 
rithm for its practical implementation. 

Let m denote a message vector, and c denote the corresponding code vector applied 
by the encoder to the input of a discrete memoryless channel. Let r denote the received 
vector, which may differ from the transmitted code vector due to channel noise. Given the 
received vector r, the decoder is required to make an estimate rh of the message vector. 
Since there is a one-to-one correspondence between the message vector m and the code 
vector c, the decoder may equivalently produce an estimate £ of the code vector. We may 
then put ni = m if and only if c = c. Otherwise, a decoding error is committed in the 
receiver. The decoding rule for choosing the estimate c, given the received vector r, is said 
to be optimum when the probability of decoding error is minimized. From the material 
presented in Chapter 6, we may state that for equiprobable messages, the probability of 
decoding error is minimized if the estimate c is chosen to maximize the log-likelihood 
function. Let p{r | c) denote the conditional probability of receiving r, given that c was sent. 
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The log-likelihood function equals log p(r | c). The maximum likelihood decoder or deci- 
sion rule is described as follows: 

Choose the estimate c for which the ^g, 
log-likelihood function log p(t | c) is maximum. 

Consider now the special case of a binary symmetric channel. In this case, both the 
transmitted code vector c and the received vector r represent binary sequences of length 
N, say. Naturally, these two sequences may differ from each other in some locations be- 
cause of errors due to channel noise. Let c, and r, denote the rth elements of c and r, 
respectively. We then have 

N 

p(r\c) = np(r,k,) (10.57) 

i-l 

Correspondingly, the log-likelihood is 

N 

logp(r|c) = 2 logp^lc) (10.58) 

i=l 

Let the transition probability pfo | Cf) be defined as 

P(r,\0 = \ P : n * ri * Ci (10.59) 
U - p, if r, = c, 

Suppose also that the received vector r differs from the transmitted code vector c in exactly 
d positions. The number d is the Hamming distance between vectors r and c. Then, we 
may rewrite the log-likelihood function in Equation (10.58) as 

log p(r|c) = d log p + (N - d) log(l - p) 

(10.60) 



In general, the probability of an error occurring is low enough for us to assume p < 1/2. 
We also recognize that N log(l - p) is a constant for all c. Accordingly, we may restate 
the maximum-likelihood decoding rule for the binary symmetric channel as follows: 

Choose the estimate c that minimizes the Hamming distance (10 61) 
between the received vector r and the transmitted vector c. 

That is, for the binary symmetric channel, the maximum-likelihood decoder reduces to a 
minimum distance decoder. In such a decoder, the received vector r is compared with each 
possible transmitted code vector c, and the particular one closest to r is chosen as the 
correct transmitted code vector. The term "closest" is used in the sense of minimum num- 
ber of differing binary symbols (i.e., Hamming distance) between the code vectors under 
investigation. 



a The Viterbi Algorithm 9 

The equivalence between maximum likelihood decoding and minimum distance decoding 
for a binary symmetric channel implies that we may decode a convolutional code by choos- 
ing a path in the code tree whose coded sequence differs from the received sequence in the 
fewest number of places. Since a code tree is equivalent to a trellis, we may equally limit 
our choice to the possible paths in the trellis representation of the code. The reason for 
preferring the trellis over the tree is that the number of nodes at any level of the trellis 
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does not continue to grow as the number of incoming message bits increases; rather it 
remains constant at 2 K ~\ where K is the constraint length of the code. 

Consider, for example, the trellis diagram of Figure 10.15 for a convolutional code 
with rate r = 1/2 and constraint length K = 3. We observe that at level /' = 3, there are 
two paths entering any of the four nodes in the trellis. Moreover, these two paths will be 
identical onward from that point. Clearly, a minimum distance decoder may make a de- 
cision at that point as to which of those two paths to retain, without any loss of perfor- 
mance. A similar decision may be made at level / = 4, and so on. This sequence of decisions 
is exactly what the Viterbi algorithm does as it walks through the trellis. The algorithm 
operates by computing a metric or discrepancy for every possible path in the trellis. The 
metric for a particular path is defined as the Hamming distance between the coded sequence 
represented by that path and the received sequence. Thus, for each node (state) in the trellis 
of Figure 10.15 the algorithm compares the two paths entering the node. The path with 
the lower metric is retained, and the other path is discarded. This computation is repeated 
for every level /' of the trellis in the range M ;' £ L, where M = K - 1 is the encoder's 
memory and L is the length of the incoming message sequence. The paths that are retained 
by the algorithm are called survivor or active paths. For a convolutional code of constraint 
length K= 3, for example, no more than 2 K ~ l = 4 survivor paths and their metrics will 
ever be stored. This list of 2 K ~ 1 paths is always guaranteed to contain the maximum- 
likelihood choice. 

A difficulty that may arise in the application of the Viterbi algorithm is the possibility 
that when the paths entering a state are compared, their metrics are found to be identical. 
In such a situation, we make the choice by flipping a fair coin (i.e., simply make a guess). 

In summary, the Viterbi algorithm is a maximum-likelihood decoder, which is op- 
timum for an AWGN channel. It proceeds in a step-by-step fashion as follows: 

Initialization 

Label the left-most state of the trellis (i.e., the all-zero state at level 0) as 0, since 
there is no discrepancy at this point in the computation. 

Computation step j + 1 

Let / = 0, 1, 2, . . . , and suppose that at the previous step / we have done two things: 
*■ All survivor paths are identified. 

* The survivor path and its metric for each state of the trellis are stored. 

Then, at level (clock time) / + 1, compute the metric for all the paths entering each state 
of the trellis by adding the metric of the incoming branches to the metric of the connecting 
survivor path from level /. Hence, for each state, identify the path with the lowest metric 
as the survivor of step ;' + 1, thereby updating the computation. 

Final Step 

Continue the computation until the algorithm completes its forward search through 
the trellis and therefore reaches the termination node (i.e., all-zero state), at which time it 
makes a decision on the maximum likelihood path. Then, like a block decoder, the se- 
quence of symbols associated with that path is released to the destination as the decoded 
version of the received sequence. In this sense, it is therefore more correct to refer to the 
Viterbi algorithm as a maximum likelihood sequence estimator. 

However, when the received sequence is very long (near infinite), the storage require- 
ment of the Viterbi algorithm becomes too high, and some compromises must be made. 
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The approach usually taken is to "truncate" the path memory of the decoder as described 
here. A decoding window of length i is specified, and the algorithm operates on a corre- 
sponding frame of the received sequence, always stopping after i steps. A decision is then 
made on the "best" path and the symbol associated with the first branch on that path is 
released to the user. The symbol associated with the last branch of the path is dropped. 
Next, the decoding window is moved forward one time interval, and a decision on the 
next code frame is made, and so on. The decoding decisions made in this way are no longer 
truly maximum likelihood, but they can be made almost as good provided that the decod- 
ing window is long enough. Experience and analysis have shown that satisfactory results 
are obtained if the decoding window length I is on the order of 5 times the constraint 
length K of the convolutional code or more. 

► Example 10.6 Correct Decoding of Received All-Zero Sequence 

Suppose that the encoder of Figure 10.13a generates an all-zero sequence that is sent over a 
binary symmetric channel, and that the received sequence is (0100010000 . . .). There are two 
errors in the received sequence due to noise in the channel: one in the second bit and the other 
in the sixth bit. "We wish to show that this double-error pattern is correctable through the 
application of the Viterbi decoding algorithm. 

In Figure 10.17, we show the results of applying the algorithm for level / = 1, 2, 3, 4, 
5. We see that for / — 2 there are (for the first time) four paths, one for each of the four states 
of the encoder. The figure also includes the metric of each path for each level in the 
computation. 

In the left side of Figure 10.17, for / = 3 we show the paths entering each of the states, 
together with their individual metrics. In the right side of the figure, we show the four survivors 
that result from application of the algorithm for level / = 3, 4, 5. 

Examining the four survivors in Figure 10.17 for / = 5, we see that the all-zero path 
has the smallest metric and will remain the path of smallest metric from this point forward. 
This clearly shows that the all-zero sequence is the maximum likelihood choice of the Viterbi 
decoding algorithm, which agrees exactly with the transmitted sequence. -4. 

► Example 10.7 Incorrect Decoding of Received All-Zero Sequence 

Suppose next that the received sequence is (1100010000 . . .), which contains three errors 
compared to the transmitted all-zero sequence. 

In Figure 10.18, we show the results of applying the Viterbi decoding algorithm for ;" = 
1, 2, 3, 4. We see that in this example the correct path has been eliminated by level / = 3. 
Clearly, a triple-error pattern is uncorrectable by the Viterbi algorithm when applied to a 
convolutional code of rate 1/2 and constraint length K = 3. The exception to this rule is a 
triple-error pattern spread over a time span longer than one constraint length, in which case 
it is very likely to be correctable. ^ 

s Free Distance of a Convolutional Code 

The performance of a convolutional code depends not only on the decoding algorithm 
used but also on the distance properties of the code. In this context, the most important 
single measure of a convolutional code's ability to combat channel noise is the free distance, 
denoted by d !rcc . The free distance of a convolutional code is defined as the minimum 
Hamming distance between any two code words in the code. A convolutional code with 
free distance d lm can correct t errors if and only if d lm is greater than It. 

The free distance can be obtained quite simply from the state diagram of the con- 
volutional encoder. Consider, for example, Figure 10.16b, which shows the state diagram 
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Figure 10.17 Illustrating steps in the Viterbi algorithm for Example 10.6. 
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of the encoder of Figure 10.13a. Any nonzero code sequence corresponds to a complete 
path beginning and ending at the 00 state (i.e., node a). We thus find it useful to split this 
node in the manner shown in the modified state diagram of Figure 10.19, which may be 
viewed as a signal-flow graph with a single input and a single output. A signal-flow graph 
consists of nodes and directed branches; it operates by the following rules: 

1. A branch multiplies the signal at its input node by the transmittance characterizing 
that branch. 

2. A node with incoming branches sums the signals produced by all of those branches. 

3. The signal at a node is applied equally to all the branches outgoing from that node. 

4. The transfer function of the graph is the ratio of the output signal to the input signal. 
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FIGURE 10.19 Modified state diagram of convolutional encoder. 



Returning to the signal-flow graph of Figure 10.19, we note that the exponent of D on a 
branch in this graph describes the Hamming weight of the encoder output corresponding 
to that branch. The exponent of L is always equal to one, since the length of each branch 
is one. Let T(D, L) denote the transfer function of the signal-flow graph, with D and L 
playing the role of dummy variables. For the example of Figure 10.19, we may readily use 
rules 1, 2, and 3 to obtain the following input-output relations: 



(10.62) 



b = D 2 La 0 + Lc 
c = DLb + DLd 
d = DLb + DLd 
a 1 = D 2 Lc 

where a 0 , b, c, d, and a x denote the node signals of the graph. Solving the set of Equations 
(10.62) for the ratio aJa Q , we find that the transfer function of the graph in Figure 10.19 
is given by 

n 5 r 3 

T(D L) = — (10.63) 

1 ' ' l-r>L(l + L) 

Using the binomial expansion, we may equivalently write 

T(D, L) = D 5 L 3 E (DL(1 + L))> (10.64) 

i-0 

Setting L = 1 in Equation (10.64), we thus get the distance transfer function expressed in 
the form of a power series: 

T(D, 1) = D 5 + ID 6 + AD 7 + ■■■ (10.65) 

Since the free distance is the minimum Hamming distance between any two code words 
in the code and the distance transfer function T(D, 1) enumerates the number of code 
words that are a given distance apart, it follows that the exponent of the first term in the 
expansion of T(D, 1) defines the free distance. Thus, on the basis of Equation (10.65), the 
convolutional code of Figure 10.13a has a free distance d im = 5. 

This result indicates that up to two errors in the received sequence are correctable, 
for two or fewer transmission errors will cause the received sequence to be at most at a 
Hamming distance of 2 from the transmitted sequence but at least at a Hamming distance 
of 3 from any other code sequence in the code. In other words, in spite of the presence ot 
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any pair of transmission errors, the received sequence remains closer to the transmitted 
sequence than any other possible code sequence. However, this statement is no longer true 
if there are three or more closely spaced transmission errors in the received sequence. These 
observations confirm the results reported earlier in Examples 10.6 and 10.7. 

In using the distance transfer function T{D, 1) to calculate the free distance of a 
convolutional code, it is assumed that the power series in the unit-delay variable D rep- 
resenting T(D, 1) is convergent (i.e., its sum has a "finite" value). This assumption is 
required to justify the expansion given in Equation (10.65) for the convolutional code of 
Figure 10.13a. However, there is no guarantee that T(D, 1) is always convergent. When 
T(D, 1) is nonconvergent, an infinite number of decoding errors are caused by a finite 
number of transmission errors; the convolutional code is then subject to catastrophic error 
propagation, and the code is called a catastrophic code? 0 In this context it is noteworthy 
that a systematic convolutional code cannot be catastrophic. Unfortunately, for a pre- 
scribed constraint length K, the free distances that can be attained with systematic con- 
volutional codes using schemes such as those shown in Figure 10.13 are usually smaller 
than for the case of nonsystematic convolutional codes, as indicated in Table 10.8. 

■ Asymptotic Coding Gain 11 

The transfer function of the encoder state diagram, modified in a manner similar to that 
illustrated in Figure 10.19, may be used to evaluate a bound on the bit error rate for a 
given decoding scheme; details of this evaluation are, however, beyond the scope of our 
present discussion. Here we simply summarize the results for two special channels, namely, 
the binary symmetric channel and the binary-input additive white Gaussian noise (AWGN) 
channel, assuming the use of binary phase-shift keying (PSK) with coherent detection. 

1. Binary symmetric channel. The binary symmetric channel may be modeled as an ad- 
ditive white Gaussian noise channel with binary phase-shift keying (PSK) as the modula- 
tion and with hard-decision demodulation. The transition probability p of the binary sym- 
metric channel is then equal to the bit error rate (BER) for the uncoded binary PSK system. 
From Chapter 6 we recall that for large values of E h /N 0 , the ratio of signal energy per bit- 
to-noise power spectral density, the bit error rate for binary PSK without coding is dom- 
inated by the exponential factor exp(-£ fc /N 0 ). On the other hand, the bit error rate for 
the same modulation scheme with convolutional coding is dominated by the exponential 
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factor exp(-d !rce rE b /2N 0 ), where r is the code rate and d frec is the free distance of the 
convolutional code. Therefore, as a figure of merit for measuring the improvement in error 
performance made by the use of coding with hard-decision decoding, we may use the 
exponents to define the asymptotic coding gain (in decibels) as follows: 

G a = 10 lofto^r) dB (10.66) 

2 Binary-input AWGN channel Consider next the case of a memoryless binary-input 
AWGN channel with no output quantization [i.e., the output amplitude lies in the interval 
(-co co)]. For this channel, theory shows that for large values of E h /N 0 the bit error rate 
for binary PSK with convolutional coding is dominated by the exponential factor 
exp(-d £[ee rE h /N 0 ), where the parameters are as previously defined. Accordingly, in this 
case, we'find that the asymptotic coding gain is denned by 

G„ = 10 log 10 (<W) dB (10.67) 

From Equations (10.66) and (10.67) we see that the asymptotic coding gain for the 
binary-input AWGN channel is greater than that for the binary symmetric channel by 3 
dB In other words, for large E„/N 0 , the transmitter for a binary symmetric channel must 
generate an additional 3 dB of signal energy (or power) over that for a binary-input AWGN 
channel if we are to achieve the same error performance. Clearly, there is an advantage to 
be gained by permitting an unquantized demodulator output instead of making hard de- 
cisions. This improvement in performance, however, is attained at the cost of increased 
decoder complexity due to the requirement for accepting analog inputs. 

The asymptotic coding gain for a binary-input AWGN channel is approximated to 
within about 0.25 dB by a binary input Q-ary output discrete memoryless channel with 
the number of representation levels Q = 8. This means that we may avoid the need for 
an analog decoder by using a soft-decision decoder that performs finite output quantization 
(typically, Q = 8), and yet realize a performance close to the optimum. 
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In the traditional approach to channel coding described in the preceding sections of the 
chapter, encoding is performed separately from modulation in the transmitter; likewise tor 
decoding and detection in the receiver. Moreover, error control is provided by transmitting 
additional redundant bits in the code, which has the effect of lowering the information bit 
rate per channel bandwidth. That is, bandwidth efficiency is traded for increased power 

efficiency^^ ^ effective utilization of the available bandwidth and power, coding 
and modulation have to be treated as a single entity. We may deal with this new situation 
by redefining coding as the process of imposing certain patterns on the transmitted signal. 
indeed, this definition includes the traditional idea of parity coding. 

Trellis codes for band-limited channels result from the treatment of modulation ana 
coding as a combined entity rather than as two separate operations. The combination itselr 
is referred to as trellis-coded modulation (TCM). This form of signaling has three basic 
features: 

1 The number of signal points in the constellation used is larger than what is required 
" for the modulation format of interest with the same data rate; the additional points 
allow redundancy for forward error-control coding without sacrificing bandwidth. 
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2. Convolutional coding is used to introduce a certain dependency between successive 
signal points, such that only certain patterns or sequences of signal points are 
permitted. 

3. Soft-decision decoding is performed in the receiver, in which the permissible sequence 
of signals is modeled as a trellis structure; hence, the name "trellis codes." 

This latter requirement is the result of using an enlarged signal constellation. By increasing 
the size of the constellation, the probability of symbol error increases for a fixed signal- 
to-noise ratio. Hence, with hard-decision demodulation we would face a performance loss 
before we begin. Performing soft-decision decoding on the combined code and modulation 
trellis ameliorates this problem. 

In the presence of AWGN, maximum likelihood decoding of trellis codes consists of 
finding that particular path through the trellis with minimum squared Euclidean distance 
to the received sequence. Thus, in the design of trellis codes, the emphasis is on maximizing 
the Euclidean distance between code vectors (or, equivalently, code words) rather than 
maximizing the Hamming distance of an error-correcting code. The reason for this ap- 
proach is that, except for conventional coding with binary PSK and QPSK, maximizing 
the Hamming distance is not the same as maximizing the squared Euclidean distance. 
Accordingly, in what follows, the Euclidean distance is adopted as the distance measure 
of interest. Moreover, while a more general treatment is possible, the discussion is (by 
choice) confined to the case of two-dimensional constellations of signal points. The im- 
plication of such a choice is to restrict the development of trellis codes to multilevel am- 
plitude and/or phase modulation schemes such as M-ary PSK and M-ary QAM. 

The approach used to design this type of trellis codes involves partitioning an M-ary 
constellation of interest successively into 2, 4, 8, . . . subsets with size M/2, M/4, MJ8, . . . , 
and having progressively larger increasing minimum Euclidean distance between their re- 
spective signal points. Such a design approach by set partitioning represents the "key idea" 
in the construction of efficient coded modulation techniques for band-limited channels. 

In Figure 10.20, we illustrate the partitioning procedure by considering a circular 
constellation that corresponds to 8-PSK. The figure depicts the constellation itself and the 
2 and 4 subsets resulting from two levels of partitioning. These subsets share the common 
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Figure 10.20 Partitioning of 8-PSK constellation, which shows that d 0 < d x < i 
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FIGURE 10.21 Partitioning of 16-QAM constellation, which shows that d 0 < ci, < d 2 < d 3 . 



property that the minimum Euclidean distances between their individual points follow an 
increasing pattern: d 0 < d^ < d 2 . 

Figure 10.21 illustrates the partitioning of a rectangular constellation corresponding 
to 16-QAM. Here again we see that the subsets have increasing within-subset Euclidean 
distances: d 0 < d-i< d 2 < d 3 . ' 

Based on the subsets resulting from successive partitioning of a two-dimensional 
constellation, we may devise relatively simple and yet highly effective coding schemes. 
Specifically, to send n bits/symbol with quadrature modulation (i.e., one that hasm-phase 
and quadrature components), we start with a two-dimensional constellation of 2" + signal 
points appropriate for the modulation format of interest; a circular grid is used for Al-ary 
PSK, and a rectangular one for M-ary QAM. In any event, the constellation is partitioned 
into 4 or 8 subsets. One or two incoming bits per symbol enter a rate-1/2 or rate-2/3 
binary convolutional encoder, respectively; the resulting two or three coded bits per symbol 
determine the selection of a particular subset. The remaining uncoded data bits determine 
which particular point from the selected subset is to be signaled. This class of trellis codes 
is known as Ungerboeck codes. 

Since the modulator has memory, we may use the Viterbi algorithm to perform 
maximum likelihood sequence estimation at the receiver. Each branch in the trellis of the 
Ungerboeck code corresponds to a subset rather than an individual signal point. The first 
step in the detection is to determine the signal point within each subset that is closest to 
the received signal point in the Euclidean sense. The signal point so determined and its 
metric (i.e., the squared Euclidean distance between it and the received point) may be used 
thereafter for the branch in question, and the Viterbi algorithm may then proceed m the 
usual manner. 



a Ungerboeck Codes for 8-PSK 

The scheme of Figure 10.22fl depicts the simplest Ungerboeck 8-PSK code for the trans- 
mission of 2 bits/symbol. The scheme uses a rate-1/2 convolutional encoder; the corre- 
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sponding trellis of the code is shown in Figure 10.22fo, which has four states. Note that 
the most significant bit of the incoming binary word is left uncoded. Therefore, each branch 
of the trellis may correspond to two different output values of the 8-PSK modulator or, 
equivalently, to one of the four 2-point subsets shown in Figure 10.20, The trellis of Figure 
10.22fo also includes the minimum distance path. 

The scheme of Figure 10.23a depicts another Ungerboeck 8-PSK code for trans- 
mitting 2 bits/sample; it is next in the level of complexity. This second scheme uses a 
rate-2/3 convolutional encoder. Therefore, the corresponding trellis of the code has eight 
states, as shown in Figure 10.23fo. In this case, both bits of the incoming binary word are 
encoded. Hence, each branch of the trellis corresponds to a specific output value of the 
8-PSK modulator. The trellis of Figure 10.23& also includes the minimum distance path. 

Figures \Q.22b and 10.23fc also include the encoder states. In Figure 10.22, the state 
of the encoder is defined by the contents of the two-stage shift register. On the other hand, 
in Figure 10.23, it is defined by the content of the single-stage (top) shift register followed 
by that of the two-stage (bottom) shift register. 
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Figure 10.23 (a) Eight-state Ungerboeck code for 8-PSK; the mapper follows Figure 10.20. 
(k) Trellis of the code with only some of the branches shown. 
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a Asymptotic Coding Gain 

Following the discussion in Section 10.6, we define the asymptotic coding gain of Unger- 
boeck codes as 

G. = 10 lofeo^) (10.68) 

where d !m is the free Euclidean distance of the code and d rcl is the minimum Euclidean 
distance of an uncoded modulation scheme operating with the same signal energy per bit. 
For example, by using the Ungerboeck 8-PSK code of Figure 10.22a, the signal constel- 
lation has 8 message points, and we send 2 message bits per point. Hence, uncoded trans- 
mission requires a signal constellation with 4 message points. We may therefore regard 
uncoded 4-PSK as the reference for the Ungerboeck 8-PSK code of Figure 10.22a. 

The Ungerboeck 8-PSK code of Figure 10.22a achieves an asymptotic coding gain 
of 3 dB, calculated as follows: 

1. Each branch of the trellis in Figure 10.22b corresponds to a subset of two antipodal 
signal points. Hence, the free Euclidean distance d !ree of the code can be no larger 
than the Euclidean distance d 2 between the antipodal signal points of such a subset. 
We may therefore write 

<4« = d 2 = 2 

where the distance d 2 is defined in Figure 10.24a; see also Figure 10.20. 

2. The minimum Euclidean distance of an uncoded QPSK, viewed as a reference op- 
erating with the same signal energy per bit, equals (see Figure 10.24b) 

4* = V2 

Hence, as previously stated, the use of Equation (10.68) yields an asymptotic coding gain 
of 10 log 10 2 = 3 dB. 

The asymptotic coding gain achievable with Ungerboeck codes increases with the 
number of states in the convolutional encoder. Table 10.9 presents the asymptotic coding 
gain (in dB) for Ungerboeck 8-PSK codes for increasing number of states, expressed with 
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FIGURE 10.24 Signal-space diagrams for calculation of asymptotic coding gain of Ungerboeck 
8-PSK code, (a) Definition of distance d 2 . {b) Definition of reference distance d„ f . 
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TABLE 10.9 Asymptotic coding gain of Ungerboeck 8-PSK codes, 



Number of states 


,4 


8 


16 


32 


64 


128 


256 


512 


Coding gain (dB) 


3 


3.6 


4.1 


4.6 


4.8 


5 


5.4 


5.7 



respect to uncoded 4-PSK. Note that improvements on the order of 6 dB require codes 
with a very large number of states. 
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Traditionally, the design of good codes has been tackled by constructing codes with a great 
deal of algebraic structure, for which there are feasible decoding schemes. Such an ap- 
proach is exemplified by the linear block codes and convolutional codes discussed in pre- 
ceding sections. The difficulty with these traditional codes is that, in an effort to approach 
the theoretical limit for Shannon's channel capacity, we need to increase the code-word 
length of a linear block code or the constraint length of a convolutional code, which, in 
turn, causes the computational complexity of a maximum likelihood decoder to increase 
exponentially. Ultimately, we reach a point where complexity of the decoder is so high 
that it becomes physically unrealizable. 

Various approaches have been proposed for the construction of powerful codes with 
large "equivalent" block lengths structured in such a way that the decoding can be split 
into a number of manageable steps. Building on these previous approaches, the develop- 
ment of turbo codes and low-density parity-check codes has been by far most successful. 
Indeed, this development has opened a brand new and exciting way of constructing good 
codes and decoding them with feasible complexity. Turbo codes are discussed in this sec- 
tion and low-density parity-check codes are discussed in Section 10.10. 

a Turbo Coding 

In its most basic form, the encoder of a turbo code consists of two constituent systematic 
encoders joined together by means of an interleaver, as illustrated in Figure 10.25. 

An interleaver is an input-output mapping device that permutes the ordering of a 
sequence of symbols from a fixed alphabet in a completely deterministic manner; that is, 
it takes the symbols at the input and produces identical symbols at the output but in a 
different temporal order. The interleaver can be of many types, of which the periodic and 
pseudo-random are two. Turbo codes use a pseudo-random interleaver, which operates 
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Figure 10.25 Block diagram of turbo encoder. 
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only on the systematic bits. There are two reasons for the use of an interleaver in a turbo 
code: 

► To tie together errors that are easily made in one half of the turbo code to errors 
that are exceptionally unlikely to occur in the other half. This is indeed the main 
reason why the turbo code performs better than a traditional code. 

> To provide robust performance with respect to mismatched decoding, which is a 
problem that arises when the channel statistics are not known or have been incor- 
rectly specified. 

Typically, but not necessarily, the same code is used for both constituent encoders 
in Figure 10.25. The constituent codes recommended for turbo codes are short constraint- 
length recursive systematic convolutional (RSC) codes. The reason for making the con- 
ventional codes recursive {i.e., feeding one or more of the tap outputs in the shift register 
back to the input) is to make the internal state of the shift register depend on past outputs. 
This affects the behavior of the error patterns (a single error in the systematic bits produces 
an infinite number of parity errors), with the result that a better performance of the overall 
coding strategy is attained. 



I> Example 10.8 Eight-state RSC Encoder 

Figure 10.26 shows an example eight-state RSC encoder. The generator matrix for this re- 
cursive convolutional code is 



,™ L 1 + D + D 2 + D 3 



(10.69) 



where D is the delay variable. The second entry of the matrix g(D) is the transfer function of 
the feedback shift register, denned as the transform of the output divided by the transform 
of the input. Let M(D) denote the transform of the message sequence [m l }f -1 and B(D) denote 
the transform of the parity sequence {fc,-}".T*. By definition, we have 

B(D) = 1 + D + D 2 + D 3 
M(D) ~ 1 + D + D 3 

Cross-multiplying, we get: 

(1 + D + D 2 + D 3 )M(D) = (1 + D + D 3 )B{D) 
which, on inversion into the time domain, yields 

m, + + m,_ 2 + + fe, + + 6,_ 3 = 0 (10.70) 




Figure 10.26 Example eight-state recursive systematic convolutional (RSC) encoder. 
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where the addition is modulo-2. Equation (10.70) is the parity-check equation, which the 
convolutional encoder of Figure 10.26 satisfies at each time step <". 



In Figure 10.25 the input data stream is applied directly to encoder 1, and the pseudo- 
randomly reordered version of the same data stream is applied to encoder 2. The systematic 
bits (i.e., original message bits) and the two sets of parity-check bits generated by the two 
encoders constitute the output of the turbo encoder. Although the constituent codes are 
convolutional, in reality turbo codes are block codes with the block size being determined 
by the size of the interleaver. Moreover, since both RSC encoders in Figure 10.25 are 
linear, we may describe turbo codes as linear block codes. 

The block nature of the turbo code raises a practical issue: How do we know the 
beginning and the end of a code word? The common practice is to initialize the encoder 
to the all-zero state and then encode the data. After encoding a certain number of data 
bits a number of tail bits are added so as to make the encoder return to the all-zero state 
at the end of each block; thereafter the cycle is repeated. The termination approaches of 
turbo codes include the following: 

» A simple approach is to terminate the first RSC code in the encoder and leave the 
second one unterminated. A drawback of this approach is that the bits at the end of 
the block due to the second RSC code are more vulnerable to noise than the other 
bits. Experimental work has shown that turbo codes exhibit a leveling off in perfor- 
mance as the SNR increases. This behavior is not like an error floor, but it has the 
appearance of an error floor compared to the steep drop in error performance at low 
SNR. This error floor is affected by a number of factors, the dominant one of which 
is the choice of interleaver. 

► A more refined approach 14 is to terminate both constituent codes in the encoder in 
a symmetric manner. Through the combined use of a good interleaver and dual 
termination, the error floor can be reduced by an order of magnitude compared to 
the simple termination approach. 

In the original version of the turbo encoder, the parity-check bits generated by the 
two encoders in Figure 10.25 were punctured prior to data transmission over the channel 
to maintain the rate at 1/2. A punctured code is constructed by deleting certain parity 
check bits, thereby increasing the data rate. Puncturing is the inverse of extending a code. 
It should, however, be emphasized that the use of a puncture map is not a necessary 
requirement for the generation of turbo codes. 

The novelty of the parallel encoding scheme of Figure 10.25 is in the use of recursive 
systematic convolutional (RSC) codes and the introduction of a pseudo-random interleaver 
between the two encoders. Thus a turbo code appears essentially random to the channel 
by virtue of the pseudo-random interleaver, yet it possesses sufficient structure for the 
decoding to be physically realizable. Coding theory asserts that a code chosen at random 
is capable of approaching Shannon's channel capacity, provided that the block size is 
sufficiently large. 15 This is indeed the reason behind the impressive performance of turbo 
codes, as discussed next. 

b Performance of Turbo Codes 

Figure 10.27 shows the error performance of a 1/2 rate, turbo code with a large block size 
for binary data transmission over an AWGN channel. 16 The code uses an interleaver oi 
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E b IN 0 , dB 

FIGURE 10.27 Noise performances of 1/2 rate, turbo code and uncoded transmission for 
AWGN channel; the figure also includes Shannon's theoretical limit on channel capacity for code 
rate r = 1/2. 



size 65,536 and a BCJR-based decoder; details of this decoder are presented later in the 
section. Eighteen iterations of turbo decoding were used in the computation. 

For the purpose of comparison, Figure 10.27 also includes two other curves for the 
same AWGN channel: 

► Uncoded transmission (i.e., code rate r = 1). 

*- Shannon's theoretical limit for code rate 1/2, which follows from Figure 9.186. 
From Figure 1 0.27, we may draw two important conclusions: 

1. Although the bit error rate for the turbo-coded transmission is significantly higher 
than that for uncoded transmission at low E b /N 0 , the bit error rate for the turbo- 
coded transmission drops very rapidly once a critical value of E;,/N 0 has been 
reached. 

2. At a bit error rate of 10 -5 , the turbo code is less than 0.5 dB from Shannon's theo- 
retical limit. 

Note, however, attaining this highly impressive performance requires that the size of 
the interleaver, or, equivalently, the block length of the turbo code, be large. Also, the 
large number of iterations needed to improve performance increases the decoder latency. 
This drawback is due to the fact that the digital processing of information does not lend 
itself readily to the application of feedback, which is a distinctive feature of the turbo 
decoder. 

Now that we have an appreciation for the impressive performance of turbo codes, 
the stage is set for a discussion of how turbo decoding is actually performed. 

■ Turbo Decoding 

Turbo codes derive their distinctive name from analogy of the decoding algorithm to the 
"turbo engine" principle. Figure 10.28a shows the basic structure of the turbo decoder. It 
operates on noisy versions of the systematic bits and the two sets of parity-check bits in 
two decoding stages to produce an estimate of the original message bits. 
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Figure 10.28 (a) Block diagram of turbo decoder, (b) Extrinsic form of turbo decoder, where I 
stands for interleave^ D for de-interleaver, and BCJR for BCJR algorithm for log-MAP decoding. 



Each of the two decoding stages uses a BCJR algorithm, 17 which was originally 
invented by Bahl, Cocke, Jelinek, and Raviv (hence the name) to solve a maximum a 
posteriori probability (MAP) detection problem. The BCJR algorithm differs from the 
Viterbi algorithm in two fundamental respects: 

1. The BCJR algorithm is a soft input-soft output decoding algorithm with two recur- 
sions, one forward and the other backward, both of which involve soft decisions. In 
contrast, the Viterbi algorithm is a soft input-hard output decoding algorithm, with 
a single forward recursion involving soft decisions; the recursion ends with a hard 
decision, whereby a particular survivor path among several ones is retained. In com- 
putational terms, the BCJR algorithm is therefore more complex than the Viterbi 
algorithm because of the backward recursion. 

2. The BCJR algorithm is a MAP decoder in that it minimizes the bit errors by esti- 
mating the a posteriori probabilities of the individual bits in a code word; to recon- 
struct the original data sequence, the soft outputs of the BCJR algorithm are hard- 
limited. On the other hand, the Viterbi algorithm is a maximum likelihood sequence 
estimator in that it maximizes the likelihood function for the whole sequence, not 
each bit. As such, the average bit error rate of the BCJR algorithm can be slightly 
better than the Viterbi algorithm; it is never worse. 

Most important, formulation of the BCJR algorithm rests on the fundamental assumptions 
that (1) the channel encoding, namely, the convolutional encoding performed in the trans- 
mitter, is modeled as a Markov process, and (2) the channel is memoryless. In the context 
of our present discussion, the Markovian assumption means that if a code can be repre- 
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sented as a trellis, then the present state of the trellis depends only on the past state and 
the input bit. (A mathematical treatment of the BCJR algorithm is given later in this 
section.) 

Before proceeding to describe the operation of the two-stage turbo decoder in Figure 
10.28a, we find it desirable to introduce the notion of extrinsic information. The most 
convenient representation for this concept is as a log-likelihood ratio, in which case ex- 
trinsic information is computed as the difference between two log-likelihood ratios as 
depicted in Figure 10.29. Formally, extrinsic information, generated by a decoding stage 
for a set of systematic (message) bits, is defined as the difference between the log-likelihood 
ratio computed at the output of that decoding stage and the intrinsic information repre- 
sented by a log-likelihood ratio fed back to the input of the decoding stage. In effect, 
extrinsic information is the incremental information gained by exploiting the dependencies 
that exist between a message bit of interest and incoming raw data bits processed by the 
decoder. 

On this basis, we may depict the flow of information in the two-stage turbo decoder 
of Figure 10.28fl in a symmetric extrinsic manner as shown in Figure 10.286. The first 
decoding stage uses the BCJR algorithm to produce a soft estimate of systematic bit x jy 
expressed as the log-likelihood ratio 

\P(Xf = o|u, r, l5 / 2 (x))/ 
where u is the set of noisy systematic bits, £j is the set of noisy parity-check bits generated 
by encoder 1, and I 2 (x) is the extrinsic information about the set of message bits x derived 
from the second decoding stage and fed back to the first stage. Assuming that the k message 
bits are statistically independent, the total log-likelihood ratio at the output of the first 
decoding stage is therefore 

>i« = 2 h(*i) (10.72) 

Hence, the extrinsic information about the message bits derived from the first decoding 
stage is 

7,(x) = h(x) - «x) (10.73) 

where / 2 (x) is to be defined. 

Before application to the second decoding stage, the extrinsic information /, (x) is re- 
ordered to compensate for the psuedo-random interleaving introduced in the turbo en- 
coder. In addition, the noisy parity-check bits £ 2 generated by encoder 2 are used as input. 
Thus by using the BCJR algorithm, the second decoding stage produces a more refined 
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FIGURE 10.29 Illustrating the concept of extrinsic information. 
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soft estimate of the message bits x. This estimate is re-interleaved to produce the total log- 
likelihood ratio Z 2 (x). The extrinsic information / 2 (x) fed back to the first decoding stage 
is therefore 

/ 2 (x) = / 2 (x)- Mx) (10.74) 

where /jfx) is itself denned by Equation (10.73), and / 2 (x) is the log-likelihood ratio com- 
puted by the second stage. Specifically, for the ;'th element of the vector x, we have 

l 2 (x f ) = log 2 ? , ; - 1, 2, . . . , k (10.75) 

\P{Xj = 0|u, £ 2 , k(x))/ 

Through the application of / 2 (x) to the first stage, the feedback loop around the pair of 
decoding stages is thereby closed. Note that although in actual fact the set of noisy sys- 
tematic bits u is only applied to the first decoding stage as in Figure 10.28a, by formulating 
the information flow in the symmetric extrinsic manner depicted in Figure 10.286 we find 
that u is, in effect, also applied to the second decoding stage. 

An estimate of the message bits x is computed by hard-limiting the log-likelihood 
ratio / 2 (x) at the output of the second stage, as shown by 

x = sgn(/ 2 (x)) (10.76) 

where the signum function operates on each element of / 2 (x) individually. 

To initiate the turbo decoding algorithm, we simply set / 2 (x) = 0 on the first itera- 
tion of the algorithm; see Figure 10.286. 

The motivation for feeding only extrinsic information from one stage to the next in 
the turbo decoder of Figure 10.28 is to maintain as much statistical independence between 
the bits as possible from one iteration to the next. The feedback decoding strategy described 
herein implicitly relies on this assumption. If this assumption of statistical independence 
is strictly true, it can be shown that the estimate x defined in Equation (10.76) approaches 
the MAP solution as the number of iterations approaches infinity. 18 The assumption of 
statistical independence appears to be close to the truth in the vast majority of cases en- 
countered in practice. 



■ The BCJR Algorithm 

For a discussion of turbo decoding to be complete, a mathematical exposition of the BCJR 
algorithm for MAP estimation is in order. 

Let x(t) be the input to a trellis encoder at time t. Let y(t ) be the corresponding output 
observed at the receiver. Note that y(t) may include more than one observation; for ex- 
ample, a rate lln code produces n bits for each input bit, in which case we have an 
H-dimensional observation vector. Let the observation vector be denoted by 

ym = t^D. yw> • • • » 

Let A„(f) denote the probability that a state s(f) of the trellis encoder equals m, where 
m = 1, 2, . . . , M. We may then write 



X(f) = P[s(t)|y] 



(10.77) 
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where s(t) and X(t) are both M-by-1 vectors. Then, for a rate 1/n linear convolutional code 
with feedback as in the RSC code, the probability that a symbol "1" was the message bit 
is given by 



P(x(t) = 1 1 y) = 2 A s (0 



(10.78) 



where 3» A is the set of transitions that correspond to a symbol "1" at the input, and \ s (t) 
is the s-component of k(t). 

Define the forward estimation of state probabilities as the M-by-1 vector 



a(t) = P(s(t)\y( UI ) 



(10.79) 



where the observation vector y (1>;) is defined above. Also define the backward estimation 
of state probabilities as the M-by-1 vector 



where 



p({) = P(m\y lt ,k } ) 



ym) = b(t), y(t + i). • • • , y(k)] 



(10.80) 



The vectors a(t) and 0(t) are estimates of the state probabilities at time t based on the past 
and future data, respectively. We may then formulate the separability theorem as follows: 

The state probabilities at time t are related to the forward estimator a(f) and back- 
ward estimator fl(t) by the vector 



X(f) 



a(t) ■ P(t) 
II a(t) ■ P(t) I 



(10.81) 



where at(t) • p(f) is the veaor product of a(f) and p(f), and || a(t) • P(f) \\i is the 
L, norm of this vector product. 

The vector product a{t) ■ p(t) (not to be confused with the inner product) is defined in 
terms of the individual elements of a(t) and P(f) by 



a(t) ■ p(f) = 



<*i(t)Pi(t) 
<*i(t)Pi(t) 

_ot M (t)p M (t). 



and the L t norm of a(t) • p(f) is defined by 

M 

ll«(f) • pw lli = E 



(10.82) 



(10.83) 



The separability theorem says that the state distribution at time t given the past is 
independent of the state distribution at time t given the future, which is intuitively satisfying 
recalling the Markovian assumption for channel encoding, which is basic to the BCJR 
algorithm. Moreover, this theorem provides the basis of a simple way of combining the 
forward and backward estimates to obtain a complete description of the state probabilities. 

To proceed further, let the state transition probability at time t be defined by 



y m ;Jt) = P(s(t) =m, y(t)\s{t- 1) 



(10.84) 
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and denote the M-by-M matrix of transition probabilities as 

T« = ly m ^(t)} (10.85) 
We may then formulate the recursion theorem as follows: 

The forward estimate a{t) and backward estimate P(t) are computed recursively as 

II a'{t - l)r(t) ||i 

and 

_ r(t + i)p« + i) 
p(t) " || r(t + !)«» + Dlh (10 - 87> 

where the superscript T denotes matrix transposition. 

The separability and recursion theorems together define the BCJR algorithm for the 
computation of a posteriori probabilities of the states and transitions of a code trellis, 
given the observation vector. Using these estimates, the likelihood ratios needed for turbo 
decoding may then be computed by performing summations over selected subsets of states 
as required. 



10.9 Computer Experiments 
Turbo Decoding 

Two properties constitute the hallmark of turbo codes: 
Property 1: 

The error performance of the turbo decoder improves with the number of iterations of the 
decoding algorithm. This is achieved by feeding extrinsic information from the output of 
the first decoding stage to the input of the second decoding stage in the forward path and 
feeding extrinsic information from the output of the second stage to the input of the first 
stage in the backward path, and then permitting the iterative decoding process to take its 
natural course in response to the received noisy message and parity bits. 



Property 2 

The turbo decoder is capable of approaching the Shannon theoretical limit of channel 
capacity in a computationally feasible manner; this property has been demonstrated ex- 
perimentally but not yet proven theoretically. 

Property 2 requires that the block length of the turbo code be large. Unfortunately, a 
demonstration of this property requires the use of sophisticated implementations of the 
turbo decoding algorithm that are beyond the scope of this book. Accordingly, we focus 
our attention on a demonstration of Property 1 in this computer experiment. 

So, as the primary objective of this computer experiment, we wish to use the log- 
MAP implementation of the BCJR algorithm to demonstrate Property 1 of turbo decoding. 
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Figure 10.30 Results of the computer experiment on turbo decoding, for increasing number of 
iterations. 

The only channel impairment assumed in the experiment is additive white Gaussian noise. 
Details of the turbo encoder and decoder are as follows: 
Turbo Encoder (described in Figure 10.25): 

Encoder 1: convolutional encoder [1, 1, 1] 
Encoder 2: convolutional encoder [1, 0, 1] 
Block (i.e., interleaver) length: 1,200 bits 

Turbo Decoder (described in Figure 10.28): 
The BCJR algorithm for log-MAP decoding. 

The experiment was carried out for E b /N 0 = 1, 1.5, 2, and 2.5 dB, with varying 
number of iterations at each E h /N 0 . For each trial of the experiment, the number of bit 
errors was calculated after accumulating a total of 20 blocks of data (each 1,200 bits long) 
that were noise-corrupted. The probability of error was then evaluated as the ratio of bit 
errors to the total number of encoded bits. Note that in this calculation, many of the blocks 
of encoded bits were correctly decoded. 

The results of the experiment are plotted in Figure 1 0.30. The following observations 
can be made from this figure: 

1. For a given E b /N 0 , the probability of error decreases with increasing number of 
iterations, confirming Property 1 of turbo decoding. 

2. After eight iterations, there is no significant improvement in decoding performance. 

3. For a fixed number of iterations, the probability of error decreases with increasing 
E b /N 0 , which is to be expected. 

10. 10 Law-Density Parity-Check Codes 19 

Turbo codes, discussed in Section 10.8, and low-density parity-check (LDPC) codes, dis- 
cussed in this section, belong to a broad family of error-control coding techniques called 
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compound codes. The two most important advantages of LDPC codes over turbo codes 
are: 

*■ Absence of low-weight code words. 
> Iterative decoding of lower complexity. 

With regard to the issue of low-weight code words, we usually find that a small 
number of code words in a turbo code are undesirably close to the given code word. Due 
to this closeness Ln weights, once in a while the channel noise causes the transmitted code 
word to be mistaken for a nearby code word. Indeed, it is this behavior that is responsible 
for the error floor (typically around a bit error rate of 1(T 5 to 10~ 6 ) that was mentioned 
earlier. In contrast, LDPC codes can be easily constructed so that they do not have such 
low-weight code words, and they can therefore achieve vanishingly small bit error rates. 
The error-floor problem in turbo codes can be alleviated by careful design of the 
interleaver. 

Turning next to the issue of decoding complexity, we note that the computational 
complexity of a turbo decoder is dominated by the BCJR algorithm, which operates on 
the trellis for the convolutional code used in the encoder. The number of computations in 
each recursion of the BCJR algorithm scales linearly with the number of states in the trellis. 
Commonly used turbo codes employ trellises with 16 states or more. In contrast, LDPC 
codes use a simple parity-check trellis that has just two states. Consequently, the decoders 
for LDPC codes are significantly simpler than those for turbo decoders. Moreover, being 
parallelizable, LDPC decoding may be performed at greater speeds than turbo decoding. 

However, a practical objection to the use of LDPC codes is that for large block 
lengths, their encoding complexity is high compared to turbo codes. 

h Construction of LDPC Codes 

LDPC codes are specified by a parity-check matrix denoted by A, which is sparse; that is, 
it consists mainly of Os and a small number of Is. In particular, we speak of («, t c , t r ) 
LDPC codes, where n denotes the block length, t c denotes the weight (i.e., number of Is) 
in each column of the matrix A, and t r denotes the weight of each row with t r > t c . The 
rate of such a LDPC code is 

r = 1 - ^ (10.88) 

t, 

whose validity may be justified as follows. Let p denote the density of Is in the parity- 
check matrix A. Then, following the terminology introduced in Section 10.3, we may set 

t 0 = p{n — k) 

and 

tr = P" 

where (« - k) is the number of rows in A and n is the number of columns (i.e., the block 
length). Therefore, dividing t c by t„ we get 

t r n 
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By definition, the code rate of a block code is kin, hence the result of Equation (10.88) 
follows. For this result to hold, however, the rows of A must be linearly independent. 

The structure of LDPC codes is well portrayed by bipartite graphs. Figure 10.31 
shows such a graph for the example code of n = 10, £ c = 3, and t T = 5. The left-hand 
nodes in the graph of Figure 10.31 are variable nodes, which correspond to elements of 
the code word. The right-hand nodes of the graph are check nodes, which correspond to 
the set of parity-check constraints satisfied by code words in the code. LDPC codes of the 
type exemplified by the graph of Figure 10.31 are said to be regular in that all the nodes 
of a similar kind have exactly the same degree. In the example graph of Figure 10.31, the 
degree of the variable nodes is t c = 3, and the degree of the check nodes is t r — 5. As the 
block length n approaches infinity, each check node is connected to a vanishingly small 
fraction of variable nodes, hence the term low-density. 

The matrix A is constructed by putting Is in A at random, subject to the regularity 
constraints: 

*• Each column contains a small fixed number, t c , of Is. 
► Each row contains a small fixed number, t„ of Is. 

In practice, these regularity constraints are often violated slightly in order to avoid having 
linearly dependent rows in the parity-check matrix A. 

Unlike the linear block codes discussed in Section 10.3, the parity-check matrix A of 
LDPC codes is not systematic (i.e., it does not have the parity-check bits appearing in 
diagonal form), hence the use of a symbol different from that used in Section 10.3. Nev- 
ertheless, for coding purposes, we may derive a generator matrix G for LDPC codes by 
means of Gaussian elimination performed in modulo-2 arithmetic; this procedure is illus- 
trated later in Example 10.9. Following the terminology introduced in Section 10.3, the 
1-by-n code vector c is first partitioned as 

c = [b • m] 
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FIGURE 10.31 Bipartite graph of the (10, 3, 5) LDPC code. 
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where m is the k-by-1 message vector, and b is the («-fe)-by-l parity vector; see Equation 
(10.9). Correspondingly, the parity-check matrix A is partitioned as 



(10.89) 



where A t is a square matrix of dimensions (n-k) X (n-k), and A 2 is a rectangular matrix 
of dimensions k X [n-k); transposition symbolized by the superscript T is used in the 
partitioning of matrix A for convenience of presentation. Imposing the constraint of Equa- 
tion (10.16) on the LDPC code, we may write 



[b • m] 



'A a ' 



= 0 



(10.90) 



or, equivalently, 

bA t + mA 2 = 0 

Recall from Equation (10.7) that the vectors m and b are related by 

b = mP 

where P is the coefficient matrix. Hence, substituting this relation into Equation (10.90), 
we readily find that, for any nonzero message vector m, the coefficient matrix of LDPC 
codes satisfies the condition 

PAj + A 2 = 0 

which holds for all nonzero message vectors and, in particular, for m in the form 
[0 • ■ • 0 1 0 ■ • • 0] that will isolate individual rows of the generator matrix. 
Solving this equation for matrix P, we get 

P = AjAr 1 (10.91) 

where Ar 1 is the inverse of matrix A l5 which is naturally defined in modulo-2 arithmetic. 
Finally, the generator matrix of LDPC codes is defined by 



g = rp • i k ] 
= [A.Ar 1 i ij 



(10.92) 



where l k is the k-by-k identity matrix; see Equation (10.12). 

It is important to note that if we take the parity-check matrix A for some arbitrary 
LDPC code and just pick (« - k) columns of A at random to form a square matrix Ai, 
there is no guarantee that A, will be nonsingular (i.e., the inverse Aj 1 will exist), even if 
the rows of A are linearly independent. In fact, for a typical LDPC code with large block 
length n, such a randomly selected A, is highly unlikely to be nonsingular, because it is 
very likely that at least one row of A t will be all 0s. Of course, when the rows of A are 
linearly independent, there will be some set of (n - k) columns of A that will make a 
nonsingular A l5 as illustrated in Example 10.9. For some construction methods for LDPC 
codes the first (n - k) columns of A may be guaranteed to produce a nonsingular A a , or 
at least do so with a high probability, but that is not true in general. 
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* Example 10.9 (10, 3, 5) LDPC Code 

Consider the bipartite graph of Figure 10.31 pertaining to a (10, 3, 5) LDPC code. The parity- 
check matrix of the code is defined by 
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which appears to be random, while maintaining the regularity constraints: t c = 3 and t r = 5. 
Partitioning the matrix A in the manner described in Equation (10.89): 
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To derive the inverse of matrix A l5 we first use Equation (10.90) to write 



[ bp, fci, b z , b 3 , bt, b s j 
b 



1 0' 

0 0 

1 0 

0 1 

1 0 
0 0. 



[ Hp, «!, » 2 , «3> "4, »ij 

u = mA 2 



Ai 



where we have introduced the vector u to denote the matrix product mA 2 . By using Gaussian 
elimination, the matrix Ai is transformed into lower diagonal form (i.e., all the elements above 
the main diagonal are zero), as shown by 
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This transformation is achieved by the following modulo-2 additions performed on the col- 
umns of square matrix At: 

®* Columns 1 and 2 are added to column 3. 

► Column 2 is added to column 4. 

► Columns 1 and 4 are added to column 5. 

► Columns 1, 2 and 5 are added to column 6. 

Correspondingly, the vector u is transformed as 

11 — » [Uq, B 1( M 0 + X, + « 2 , «1 + "3, «0 + «3 + «4> M 0 + «1 + «4 + «i] 

Accordingly, premultiplying the transformed matrix A t by the parity vector b, using successive 
eliminations in modulo-2 arithmetic working backwards, and putting the solutions for the 
elements of the parity vector b in terms of the elements of the vector u in matrix form, we get 



[«o, «1, "2, «3> «4, «<] 



Ll 1 



1 1' 



[b a , b,, b 2> b 3 , b 4 , b s 



Ar 1 



The inverse of matrix At is therefore 
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Ar' = 



The matrix product A 2 Ai 1 is (using the given value of A 2 and the value of Ai 1 just found) 



A 2 Ai 1 



1 0 0 

0 0 0 

0 0 1 

0 1 0 



1 1 0 

1 1 1 

1 1 0 

1 1 0 



Finally, using Equation (10.92), the generator of the (10, 3, 5) LDPC code is 

0 



G = 



A.Ar 1 i t 

It is important to recognize that the LDPC code described in this example is intended 
only for the purpose of illustrating the procedure involved in the generation of such a code. 
In practice, the block length n is orders of magnitude larger than that considered in this 
example. Moreover, in constructing the matrix A, we may constrain all pairs of columns to 
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have a matrix overlap (i.e., inner product of any two columns in matrix A) not to exceed 1; 
such a constraint, over and above the regularity constraints, is expected to improve the per- 
formance of LDPC codes. Unfortunately, with a small block length as that considered in this 
example, it is difficult to satisfy this additional requirement. •< 

■ Minimum Distance of LDPC Codes 

In practice, the block length of a LDPC code is large, ranging from 10 3 to 10 6 , which 
means that the number of code words in a particular code is correspondingly large. Con- 
sequently, the algebraic analysis of LDPC codes is rather difficult. It is much more pro- 
ductive to perform a statistical analysis on an ensemble of LDPC codes. Such an analysis 
permits us to make statistical statements about certain properties of member codes in the 
ensemble. Moreover, an LDPC code with these properties can be found with high prob- 
ability by a random selection from the ensemble. 

Among these properties, the minimum distance of the member codes is of particular 
interest. From Section 10.3 we recall that the minimum distance of a linear block code is, 
by definition, the smallest Hamming distance between any pair of code vectors in the code. 
Over an ensemble of LDPC codes, the minimum distance of a member code is naturally a 
random variable. Elsewhere 20 it is shown that as the block length n increases, for fixed 
t c a 3 and t r > t c the probability distribution of the minimum distance can be overbounded 
by a function that approaches a unit step function at a fixed fraction A, t of the block 
length n. Thus, for large n, practically all the LDPC codes in the ensemble have a minimum 
distance of at least «A W Table 10.10 presents the rate r and of LDPC codes for 
different values of the weight-pair (t c , t r ). From this table we see that for t c = 3 and t r = 6 
the code rate r attains its highest value of 1/2 and the fraction A, A attains its smallest value, 
hence the preferred choice of t c = 3 and t r = 6 in the design of LDPC codes. 

m Probabilistic Decoding of LDPC Codes 

At the transmitter, a message vector m is encoded into a code vector c = mG, where G is 
the generator matrix for a specified weight-pair {t c , t r ) and therefore minimum distance 
d^. The vector c is transmitted over a noisy channel to produce the received vector 

r = c + e 

where e is the error vector due to channel noise; see Equation (10.17). By construction, 
the matrix A is a parity matrix of the LDPC code; that is, AG r = 0. Given the received 

Table 10.10" The rate r and 



' fractional term At ctr of LDPC codes 
■■ for varying weights t c and t r 
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"Adapted from Gallager (1962) with permission of the IEEE. 
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vector r, the bit-by-bit decoding problem is to find the most probable vector c that satisfies 
the condition £A T = 0. 

In what follows, a bit refers to an element of the received vector r, and a check refers 
to a row of matrix A. Let denote the set of bits that participate in check i. Let 3>(j) 
denote the set of checks in which bit ;' participates. A set that excludes bit / is denoted 
by Likewise, a set that excludes check i is denoted by $(j)\i. 

The decoding algorithm has two alternating steps: horizontal step and vertical step, 
which run along the rows and columns of matrix A, respectively. In the course of these 
steps, two probabilistic quantities associated with nonzero elements of matrix A are alter- 
nately updated. One quantity, denoted by ?% defines the probability that bit / is symbol 
x (i.e., symbol 0 or 1 ), given the information derived via checks performed in the horizontal 
step, except for check i. The second quantity, denoted by Qfy, defines the probability that 
check i is satisfied, given that bit ;' is fixed at the value x and the other bits have the 
probabilities ? w : j' £ 

The LDPC decoding algorithm then proceeds as follows: 2 ' 

Initialization 

The variables P° and ?\ are set equal to the a priori probabilities pf and pf of symbols 
0 and 1, respectively, with pf + pj = 1. 

Horizontal Step 

In the horizontal step of the algorithm, we run through the checks i. Define 



AP .. = P9. 



>o — pi 
ij 1 Ij 



For each weight-pair (/, /), compute 



AQ, ; = II AP, r 



Hence, set 



Qi 



\ (1 + AQi,) 
\ (i - AG,) 



Vertical Step 



In the vertical step of the algorithm, the values of the probabilities P? and T\ are 
updated using the quantities computed in the horizontal step. In particular, for each bit;, 
compute 



p° = n . Qh 



n = °*} n Qh 



where the scaling factor a ti is chosen to make 



p? + A = i 
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In the vertical step, we may also update the pseudo-posterior probabilities: 

p° = «tf n Qg 
= n 

where a f is chosen to make 

Pf + Pj = 1 

The quantities obtained in the vertical step are used to compute a tentative estimate 
c. If the condition cA T = 0 is satisfied, the decoding algorithm is terminated. Otherwise, 
the algorithm goes back to the horizontal step. If after some maximum number of iterations 
(e.g., 100 or 200) there is no valid decoding, a decoding failure is declared. The decoding 
procedure described herein is a special case of the general low-complexity sum-product 
algorithm. 

Simply stated, the sum-product algorithm passes probabilistic quantities between the 
check nodes and variable nodes of the bipartite graph. By virtue of the fact that each 
parity-check constraint can be represented by a simple convolutional coder with one bit 
of memory, we find that LDPC decoders axe simpler to implement than turbo decoders, 
as stated earlier. 

In terms of performance, however, we may say the following in light of experimental 
results reported in the literature: Regular LDPC codes do not appear to come as close to 
Shannon's limit as do their turbo code counterparts. 



10.11 Irregular Codes 

The turbo codes discussed in Section 10.8 and the LDPC codes discussed in Section 10.10 
are both regular codes, each in its own individual way. The error-correcting performance 
of both of these codes over a noisy channel can be improved substantially by using their 
respective irregular forms. 

In a standard turbo code with its encoder as shown in Figure 10.25, the interleaver 
maps each systematic bit to a unique input bit of convolutional encoder 2. In contrast, 
irregular turbo codes 22 use a special design of interleaver that maps some systematic bits 
to multiple input bits of the convolutional encoder. For example, each of 10 percent of 
the systematic bits may be mapped to eight inputs of the convolutional encoder instead of 
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Figure 10.32 Block diagram of irregular turbo encoder. 
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a single one. As shown in Figure 10.32, similar irregular interleavers are used in both 
convolutional encoding paths to generate the parity-check bits z, and z 2 in response to the 
message bits x. irregular turbo codes are decoded in a similar fashion to regular turbo 
codes. 

To construct an irregular LDPC code, 23 the degrees of the variable and check nodes 
in the bipartite graph are chosen according to some distribution. For example, we may 
have an irregular LDPC code with the following graphical representation; 

*■ One half of the variable nodes have degree 5 and the other half of the variable nodes 
have degree 3. 

*■ One half of the check nodes have degree 6 and the other half of the check nodes 
have degree 8. 

For a given block length and a given degree sequence, we define an ensemble of codes by 
choosing the edges (i.e., the connections between the variable and check nodes) in a ran- 
dom fashion. Specifically, the edges emanating from the variable nodes are enumerated in 
some arbitrary order, and likewise for the edges emanating from the check nodes. 
Figure 10.33 plots the error performances of the following codes: 24 

► Irregular LDPC code: k = 50,000, n = 100,000, rate = 1/2 

► Turbo code (regular): k = 65,536, n = 131,072, and rate = 1/2 

► Irregular turbo code: k = 65,536, n = 131 ,072, and rate = 1/2 

where k is the number of message bits and n is the block length. The generator polynomials 
for the two convolutional encoders in the regular/irregular turbo codes are as follows: 



Encoder 1: g(D) = 1 + D 4 
Encoder 2: g(D) = 1 + D 



D 



D 3 + D 4 



Figure 10.33 also includes the corresponding theoretical limit on channel capacity for code 
rate r = 1/2. 
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FIGURE 10.33 Noise performances of regular turbo code, irregular turbo code and irregular 
low-density parity-check (LDPC) code, compared to the Shannon limit for code rate r = 1/2. 
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Based on the results presented in Figure 10.33, we may make the following 
observations: 

*■ The irregular LDPC code outperforms the regular turbo code in that it comes closer 

to Shannon's theoretical limit by 0.175 dB. 
*■ Among the three codes displayed therein, the irregular turbo code is the best in that 

it is just 0.213 dB away from Shannon's theoretical limit. 

1 0 « 1 2 Summary and Discussion 



In this chapter, we studied error-control coding techniques that have established themselves 
as indispensable tools for reliable digital communication over noisy channels. The effect 
of errors occurring during transmission is reduced by adding redundancy to the data prior 
to transmission in a controlled manner. The redundancy is used to enable a decoder in the 
receiver to detect and correct errors. 

Error-control coding techniques may be divided into two broadly defined families: 

1. Algebraic codes, which rely on abstract algebraic structure built into the design 
of the codes for decoding at the receiver. Algebraic codes include Hamming codes, 
maximal-length codes, BCH codes, and Reed-Solomon codes. These particular 
codes share two properties: 

Linearity property, the sum of any two code words in the code is also a code word. 
Cyclic property, any cyclic shift of a code word is also a code word in the code. 
Reed-Solomon codes are very powerful codes, capable of combatting both random 
and burst errors; they find applications in difficult environments such as deep-space 
communications and compact discs. 

2. Probabilistic codes, which rely on probabilistic methods for their decoding at the 
receiver. Probabilistic codes include trellis codes, turbo codes, and low-density parity- 
check codes. In particular, the decoding is based on one or the other of two basic 
methods, as summarized here: 

Soft input-bard output, which is exemplified by the Viterbi algorithm that performs 
maximum likelihood sequence estimation in the decoding of trellis codes. 
Soft input-soft output, which is exemplified by the BCJR algorithm that performs 
maximum a posteriori estimation on a bit-by-bit basis in the decoding of turbo codes, 
or a special form of the sum-product algorithm in the decoding of low-density parity- 
check codes. 

Trellis codes combine linear convolutional encoding and modulation to permit significant 
coding gains over conventional uncoded multilevel modulation without sacrificing band- 
width efficiency. Turbo codes and low-density parity-check codes share the following 
properties: 

Random encoding of a linear block kind. 
*■ Error performance within a hair's breadth of Shannon's theoretical limit on channel 
capacity in a physically realizable fashion. 

In practical terms, turbo codes and low-density parity-check codes have made it possible 
to achieve coding gains on the order of 10 dB, which is unmatched previously. These 
coding gains may be exploited to dramatically extend the range of digital communication 
receivers, substantially increase the bit rates of digital communication systems, or signifi- 
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cantly decrease the transmitted signal energy per symbol. These benefits have significant 
implications for the design of wireless communications and deep-space communications, 
just to mention two important applications of digital communications. Indeed, turbo codes 
have already been standardized for use on deep-space communication links and wireless 
communication systems. 
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4. The first error-correcting codes (known as Hamming codes) were invented by Hamming 
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pp. 141-183) and MacWilliams and Sloane (1977, pp. 257-293). Table 10.6 on binary 
BCH codes is adapted from Lin and Costello (1983). 

6 The Reed-Solomon codes are named in honor of their inventors: see their classic 1960 
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306) The book edited by Wicker and Bhargava (1994) contains an introduction to Reed- 
Solomon codes, a historical overview of these codes written by their inventors, Irving S. 
Reed and Gustave Solomon, and the applications of Reed-Solomon codes to the explora- 
tion of the solar system, the compact disc, automatic repeat-request protocols, and spread- 
spectrum multiple-access communications, and chapters on other related issues. 

7. Convolutional codes were first introduced, as an alternative to block codes, by P. Elias 
(1955). 

8. The term trellis was introduced by Forney (1973). 

9 In a classic paper, Viterbi (1967) proposed a decoding algorithm for convolutional codes 
that has become known as the Viterbi algorithm. The algorithm was recognized by Forney 
(1972 1973) to be a maximum likelihood decoder. Readable accounts of the Viterbi al- 
gorithm are presented in Lin and Costello (1983), Blahut (1990), and Adamek (1991). 
10. Catastrophic convolutional codes are discussed in Benedetto, Biglieri, and Castellani 
(1987). Table 10.8 is adapted from their book. 

II For details of the evaluation of asymptotic coding gain for binary symmetric and binary- 
input AWGN channels, see Viterbi and Omura (1979, pp. 242-252) and Lm and Costello 
(1983, pp. 322-329). 

12. Trellis-coded modulation was invented by G. Ungerboeck; its historical evolution is de- 
scribed in Ungerboeck (1982). Table 10.9 is adapted from this latter paper. 

Trellis-coded modulation may be viewed as a form of signal-space codings, view- 
point discussed at an introductory level in Chapter 14 of the book by Lee and Messer- 
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schmitt (1994). For an extensive treatment of trellis-coded modulation, see the books by 
Biglieri, Divsalar, McLane, and Simon (1991), and Schlegel (1997). 

13. Turbo codes were originated by C. Berrou and A. Glavieux. Work on these codes was 
motivated by two papers on error-correcting codes: Battail (1987), and Hagenauer and 
Hoecher (1989). The first description of turbo codes using heuristic arguments was pre- 
sented at a conference paper by Berrou, Glavieux, and Thitimajshima (1993); see also 
Berrou and Glavieux (1996). For reflections on the early work on turbo codes and subse- 
quent developments, see Berrou and Glavieux (1998). 

For a book on the basics of turbo codes, see Heegard and Wicker (1999). Using a 
procedure reminiscent of random coding (see Note 15), Benedetto and Montorosi (1996) 
have provided partial explanations for the impressive performance of turbo codes. 

In two independent studies reported in the papers by McEliece, MacKay, and Cheng 
(1998), and Kschischang and Frey (1998), it is shown that turbo decoding duplicates an 
algorithm in artificial intelligence due to Pearl (1982), which involves the propagation of 
belief. The term belief is another way of referring to a posteriori probability. These two 
papers have opened a new avenue of research, which links turbo decoding and learning 
machines. For an insightful discussion of turbo codes, see the book by Frey (1998). 

A pseudo-random interleaver is basic to the operation of turbo codes. Denenshgaran 
and Mondin (1999) present a systematic procedure for designing interleavers (i.e., per- 
muters) for turbo codes. 

14. The dual termination of turbo codes is discussed in Guinand and Lodge (1996). 

15. Random coding is discussed in Cover and Thomas (1991), Section 8.7. 

16. The plots presented in Fig. 10.27 follow those in Fig. 6.8 of the book by Frey (1998). 

17. In the early 1960s, Baum and Welch derived an iterative procedure for solving the param- 
eter estimation problem, hence the name Baum-Welch algorithm (Baum and Petrie (1966); 
Baum et al. (1970)). In the BC/R algorithm, named after Bahl, Cocke, Jelinek, and Raviv 
(1974), the Baum-Welch algorithm is applied to the problem of soft output, maximum 
likelihood decoding of convolutional codes. 

18. The proof that the estimate x in Eq. (10.76) approaches the MAP solution as the number 
of iterations approaches infinity is discussed in the paper by Moher and Gulliver (1998). 

19. Low-density parity-check (LDPC) codes were originally discovered by Gallager (1962, 
1963). They were rediscovered independently by MacKay and Neal (1995); see also 
MacKay (1999). 

In the 1960s and for a good while thereafter, the computers available at that time 
were not powerful enough to process the long block lengths that are needed to achieve 
excellent performance with LDPC codes, hence the lack of interest in their use for over 
twenty years. 

20. For a detailed treatment of the statement that the probability distribution of the minimum 
distance of an LDPC code approaches a unit step function of the block length for certain 
values of weight-pair (t c , f,), see Gallager (1962, 1963). 

21 . The decoding algorithm of LDPC codes described herein follows MacKay and Neal (1996, 
1997). 

22. Irregular turbo codes were invented by Frey and MacKay (1999). 

23. Irregular LDPC codes were invented independently by MaKay et al. (1999) and Richardson 
et al. (1999). 

24. The codes, whose performances are plotted in Fig. 10.34, are due to the following 
originators: 

► Regular turbo codes: Berrou and Glavieux (1996); Berrou et al. (1995). 

► Irregular turbo codes: Frey and MacKay (1999). 
* Irregular LDPC codes: Richardson et al. (1999). 
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: Problems 

Soft-Decision Coding 

10.1 Consider a binary input Q-ary output discrete memoryless channel. The channel is said 
to be symmetric if the channel transition probability p(j\ i) satisfies the condition: 

PU\0) = p(Q - 1 - / = 0,1,...,Q-1 

Suppose that the channel input symbols 0 and 1 are equally likely. Show that the channel 
output symbols are also equally likely; that is, 

P(/) = ^> ; = 0,1,...,Q-1 

10.2 Consider the quantized demodulator for binary PSK signals shown in Fig. 10.3a. The 
quantizer is a four-level quantizer, normalized as in Fig. PI 0.2. Evaluate the transition 
probabilities of the binary input-quarternary output discrete memoryless channel so 
characterized. Hence, show that it is a symmetric channel. Assume that the transmitted 
signal energy per bit is E b , and the additive white Gaussian noise has zero mean and 
power spectral density N 0 /2. 
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10.3 Consider a binary input AWGN channel, in which the binary symbols 1 and 0 are equally 
likely. The binary symbols are transmitted over the channel by means of phase-shift 
keying. The code symbol energy is £, and the AWGN has zero mean and power spectral 
density No/2. Show that the channel transition probability is given by 



P(y 1 0) = 




Linear Block and Cyclic Codes 

10.4 In a single-parity-check code, a single parity bit is appended to a block of k message bits 
(m u m 2 , ■ ■ ■ , m k ). The single parity bit bi is chosen so that the code word satisfies the 
even parity rule: 

ni! + m 2 + ■ • • + m k + b, = 0, mod 2 
For k = 3, set up the 2 k possible code words in the code defined by this rule. 

10.5 Compare the parity-check matrix of the (7, 4) Hamming code considered in Example 
10.2 with that of a (4, 1) repetition code. 
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10.6 Consider the (7, 4) Hamming code of Example 10.2. The generator matrix G and the 
parity-check matrix H of the code are described in that example. Show that these two 
matrices satisfy the condition 

HG T = 0 

10.7 (a) For the (7, 4) Hamming code described in Example 10.2, construct the eight code 

words in the dual code, 
(b) Find the minimum distance of the dual code determined in part (a). 

10.8 Consider the (5, 1) repetition code of Example 10.1. Evaluate the syndrome s for the 
following error patterns: 

(a) All five possible single-error patterns 

(b) All 10 possible double-error patterns 

10.9 For an application that requires error detection only, we may use a nonsystematic code. 
In this problem, we explore the generation of such a cyclic code. Let g(X) denote the 
generator polynomial, and m(X) denote the message polynomial. We define the code 
polynomial c(X) simply as 

c(X) = m(X)g(X) 

Hence, for a given generator polynomial, we may readily determine the code words in 
the code. To illustrate this procedure, consider the generator polynomial for a (7, 4) 
Hamming code: 

g{X) = 1 + X + X 3 

Determine the 16 code words in the code, and confirm the nonsystematic nature of the 
code. 

10.10 The polynomial 1 + X 7 has 1 + X + X 3 and 1 + X 1 + X 3 as primitive factors. In 
Example 10.3, we used 1 + X + X 3 as the generator polynomial for a (7, 4) Hamming 
code. In this problem, we consider the adoption of 1 + X 2 + X 3 as the generator 
polynomial. This should lead to a (7, 4) Hamming code that is different from the code 
analyzed in Example 10.3. Develop the encoder and syndrome calculator for the gen- 
erator polynomial: 

g{X) = 1 + X 2 + X 3 

Compare your results with those in Example 10.3. 

10.11 Consider the (7, 4) Hamming code defined by the generator polynomial 

g{X) = 1 + X + X 3 

The code word 0111001 is sent over a noisy channel, producing the received word 
0101001 that has a single error. Determine the syndrome polynomial s(X) for this re- 
ceived word, and show that it is identical to the error polynomial e(X). 

10.12 The generator polynomial of a (15, 11) Hamming code is defined by 

g(X) = 1 + X + X 4 

Develop the encoder and syndrome calculator for this code, using a systematic form for 
the code. 

10.13 Consider the (15, 4) maximal-length code that is the dual of the (15, 11) Hamming code 
of Problem 10.12. Do the following: 

(a) Find the feedback connections of the encoder, and compare your results with those 
of Table 7.1 on maximal-length codes presented in Chapter 7. 

(b) Find the generator polynomial g(X); hence, determine the output sequence assuming 
the initial state 0001. Confirm the validity of your result by cycling the initial state 
through the encoder. 
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10.14 Consider the (31, 15) Reed-Solomon code. 

(a) How many bits are there in a symbol of the code? 

(b) What is the block length in bits? 

(c) What is the minimum distance of the code? 

(d) How many symbols in error can the code correct? 

Convolutional Codes 

10 15 A convolutional encoder has a single-shift register with two stages, (i.e., constraint length 
K = 3), three modulo-2 adders, and an output multiplexer. The generator sequences of 
the encoder are as follows: 

g (1> = (1, 0, 1) 
g m = (1, h 0) 

g (3> = (1, 1, 1) 

Draw the block diagram of the encoder. 

Note: For Problems 10.16-10.23, the same message sequence 10111 ...is used so that 
we may compare the outputs of different encoders for the same input. 
10 16 Consider the rate r = 111, constraint length K = 2 convolutional encoder of Fig. P10.16. 
The code is systematic. Find the encoder output produced by the message sequence 
10111. . . . 



Modulo-2 
adder 



Input 




Output 



Figure PI O.l 6 



10 17 Figure P10.17 shows the encoder for a rate r = 1/2, constraint length K - 4 convolu- 
tional code. Determine the encoder output produced by the message sequence 10111. . . . 



Modulo-2 
adder 



Input c- 




Output 



Figure PI O.l 7 
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10.18 Consider the encoder of Fig. 10.136 for a rate r = 2/3, constraint length K = 2 con- 
volutional code. Determine the code sequence produced by the message sequence 
10111.... 

10.19 Construct the code tree for the convolutional encoder of Fig. P10.16. Trace the path 
through the tree that corresponds to the message sequence 10111 . . . , and compare the 
encoder output with that determined in Problem 10.16. 

10.20 Construct the code tree for the encoder of Fig. P10.17. Trace the path through the tree 

that corresponds to the message sequence 10111 Compare the resulting encoder 

output with that found in Problem 10.17. 

10.21 Construct the trellis diagram for the encoder of Fig. P10.1 7, assuming a message sequence 
of length 5. Trace the path through the trellis corresponding to the message sequence 
10111. . . . Compare the resulting encoder output with that found in Problem 10.17. 

10.22 Construct the state diagram for the encoder of Fig. P10.17. Starting with the all-zero 
state, trace the path that corresponds to the message sequence 10111 . . . , and compare 
the resulting code sequence with that determined in Problem 10.17. 

10.23 Consider the encoder of Fig. 10.13&. 

(a) Construct the state diagram for this encoder. 

(b) Starting from the all-zero state, trace the path that corresponds to the message se- 
quence 10111. .. . Compare the resulting sequence with that determined in Problem 
10.18. 

10.24 By viewing the minimum shift keying (MSK) scheme as a finite-state machine, construct 
the trellis diagram for MSK. (A description of MSK is presented in Chapter 6.) 

10.25 The trellis diagram of a rate- 1/2, constraint length-3 convolutional code is shown in 
Figure P10.25. The all-zero sequence is transmitted, and the received sequence is 
100010000. . . . Using the Viterbi algorithm, compute the decoded sequence. 




10.26 Consider a rate-1/2, constraint length-7 convolutional code with free distance tf ft . e =10. 
Calculate the asymptotic coding gain for the following two channels: 

(a) Binary symmetric channel 

(b) Binary input AWGN channel 

10.27 In Section 10.6 we described the Viterbi algorithm for maximum likelihood decoding of 
a convolutional code. Another application of the Viterbi algorithm is for maximum 
likelihood demodulation of a received sequence corrupted by intersymbol interference 
due to a dispersive channel. Figure P10.27 shows the trellis diagram for intersymbol 
interference, assuming a binary data sequence. The channel is discrete, described by the 
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finite impulse response (1, 0.1). The received sequence is (1.0, -0.3, -0.7, 0, . . .). Use 
the Viterbi algorithm to determine the maximum likelihood decoded version of this 
sequence. 




-1.1 

Figure P10.27 



10.28 Figure P10.28 depicts 32-QAM cross constellation. Partition this constellation into eight 
subsets. At each stage of the partitioning, indicate the within-subset (shortest) Euclidean 
distance. 



Figure P10.28 



10.29 As explained in the Introduction to this chapter, channel coding can be used to reduce 
the Ej/N 0 required for a prescribed error performance or reduce the size of the receiving 
antenna for a prescribed E b /N 0 . In this problem we explore these two practical benefits 
of coding by revisiting Example 8.2 in Chapter 8 on the downlink power calculations 
for a domestic satellite communication system. In particular, we now assume that the 
design of the downlink includes the use of a coding scheme consisting of a rate-1/2 
convolutional encoder with length K = 7 and Viterbi decoding. The coding gain of this 
scheme is 5.1 dB, assuming the use of soft quantization. Hence do the following: 

(a) Recalculate the required EiJN 0 ratio of the system. 

(b) Assuming that the required E b /N 0 ratio remains unchanged, calculate the reduction 
in the size of the receiving dish antenna that is made possible by the use of this 
coding scheme in the downlink. 

10.30 Unlike the convolutional codes considered in this chapter, we recall from Chapter 6 that 
the convolutional code used in the voiceband modem V.32 modem is nonlinear. Figure 
P10.30 shows the circuit diagram of the convolutional encoder used in this modem; it 
uses modulo-2 multiplication and gates in addition to modulo-2 additions and delays. 
Explain the reason for nonlinearity of the encoder in Fig. P10.30, and use an example 
to illustrate your explanation. 
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Input 



Mod-2 
multiplier 



Flip-flop 



Mod-2 
adder 



Figure F10.30 



Output 



Turbo Codes 

10.31 Let = plq-i and rf ] = plq z be the code rates of RSC encoders 1 and 2 in the turbo 
encoder of Fig. 10.25. Find the code rate of the turbo code. 

10.32 The feedback nature of the constituent codes in the turbo encoder of Fig. 10.25 has the 
following implication: A single bit error corresponds to an infinite sequence of channel 
errors. Illustrate this phenomenon by using a message sequence consisting of symbol 1 
followed by an infinite number of symbols 0. 

10.33 Consider the following generator matrices for rate 1/2 turbo codes: 



4-state encoder: 



8-state encoder: 



16-state encoder: 



g(D) = 
g(D) = 



1,- 



1 + D + D 1 



1 + D 
1 + D 2 



+ D 3 



1 + D + D 1 + D 
1 + D 4 



1 + D + D 1 + D 3 + D 4 



(a) Construct the block diagram for each one of these RSC encoders. 

(b) Setup the parity-check equation associated with each encoder. 

10.34 The turbo encoder of Fig. 10.25 involves the use of two RSC encoders. 

(a) Generalize this encoder to encompass a total of JVf interleaves. 

(b) Construct the block diagram of the turbo decoder that exploits the M sets of parity- 
check bits generated by such a generalization. 

10.35 Turbo decoding relies on the feedback of extrinsic information. The fundamental prin- 
ciple adhered to in the turbo decoder is to avoid feeding a decoding stage information 
that stems from the stage itself. Explain the justification for this principle in conceptual 
terms. 

10.36 Suppose a communication receiver consists of two components, a demodulator and a 
decoder. The demodulator is based on a Markov model of the combined modulator and 
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channel, and the decoder is based on a Markov model of a forward error correction 
code. Discuss how the turbo principle may be applied to construct a joint demodulator/ 
decoder for this system. 

Computer Experiment 

10.37 In this experiment we continue the investigation into turbo codes presented in Section 
10.9 by evaluating the effect of block size on the noise performance of the decoder. 
As before, the two convolutional encoders of the turbo encoder are as follows: 

Encoder 1: [1, 1, 1] 
Encoder 2: [1, 0, 1] 

The transmitted E b IN a is 1 dB. The block errors to termination are prescribed not to 
exceed 15. 

With this background information, plot the bit error rate of the turbo decoder 
versus the number of iterations for two different block (i.e., interleaver) sizes: 200 and 
400. 



Probability Theory 



A 1.1 Probabilistic Concepts 

Probability theory is rooted in phenomena that, explicitly or implicitly, can be modeled 
by an experiment with an outcome that is subject to chance. Moreover, if the experiment 
is repeated, the outcome can differ because of the influence of an underlying random 
phenomenon or chance mechanism. Such an experiment is referred to as a random exper- 
iment. For example, the experiment may be the observation of the result of tossing a fan- 
coin. In this experiment, the possible outcomes of a trial are "heads" or "tails." 

To be more precise in the description of a random experiment, we ask for three 
features: 

1. The experiment is repeatable under identical conditions. 

2. On any trial of the experiment, the outcome is unpredictable. 

3. For a large number of trials of the experiment, the outcomes exhibit statistical reg- 
ularity; that is, a definite average pattern of outcomes is observed if the experiment 
is repeated a large number of times. 



■ Relative-Frequency Approach 

Let event A denote one of the possible outcomes of a random experiment. For example, 
in the coin-tossing experiment, event A may represent "heads." Suppose that in n trials of 
the experiment, event A occurs N„{A) times. We may then assign the ratio N„(A)/n to the 
event A. This ratio is called the relative frequency of the event A. Clearly, the relative 
frequency is a nonnegative real number less than or equal to one. That is to say, 

0 == '- s 1 (Al.l) 
n 

If event A occurs in none of the trials, N„(A)/n = 0. If, on the other hand, event A occurs 
in all the n trials, N n (A)/n = 1. 

We say that the experiment exhibits statistical regularity if for any sequence of n 
trials the relative frequency N„{A)/n converges to the same limit as n becomes large. It thus 
seems natural for us to define the probability of event A as 

P{A) = U^j (A1 . 2) 

The limit shown in Equation (A1.2) should not be viewed in a mathematical sense. Rather, 
we think of Equation (A1.2) as a statement that the probability of an event is the long- 
term proportion of times that a particular event A occurs in a long sequence of trials. For 
example, in the coin-tossing experiment, we may expect that out of a million tosses of a 
fair coin, about one half of them will show up heads. 

The probability of an event is intended to represent the likelihood that a trial of the 
experiment will result in the occurrence of that event. For many engineering applications 
and games of chance, the use of Equation (A1.2) to define the probability of an event is 
acceptable. However, for many other applications, this definition is inadequate. Consider, 
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for example, the statistical analysis of the stock market: How are we to achieve repeata- 
bility of such an experiment? A more satisfying approach is to state the properties that 
any measure of probability is expected to have, postulate them as axioms, and then use 
relative-frequency interpretations to justify them. 

■ Axioms of Probability 

When we perform a random experiment, it is natural for us to be aware of the various 
outcomes that are likely to arise. In this context, it is convenient to think of an experiment 
and its possible outcomes as defining a space and its points. With the kth outcome of the 
experiment, say, we associate a point called the sample point, which we denote by s k . The 
totality of sample points corresponding to the aggregate of all possible outcomes of the 
experiment is called the sample space, which we denote by S. An event corresponds to 
either a single sample point or a set of sample points. In particular, the entire sample space 
S is called the sure event; the null set 0 is called the null or impossible event; and a single 
sample point is called an elementary event. 

Consider, for example, an experiment that involves the throw of a die. In this ex- 
periment there are six possible outcomes: the showing of one, two, three, four, five, and 
six dots on the upper face of the die. By assigning a sample point to each of these possible 
outcomes, we have a one-dimensional sample space that consists of six sample points, as 
shown in Figure Al.l. The elementary event describing the statement "a six shows" cor- 
responds to the sample point {6). On the other hand, the event describing the statement 
"an even number of dots shows" corresponds to the subset {2, 4, 6} of the sample space. 
Note that the term event is used interchangeably to describe the subset or the statement. 

We are now ready to make a formal definition of probability. A probability system 
consists of the triple: 

1. A sample space S of elementary events (outcomes). 

2. A class % of events that are subsets of S. 

3. A probability measure ?(•) assigned to each event A in the class %, which has the 
following properties: 

(i) 

P(S) = 1 (A1.3) 

(ii) 

0 < P(A) £ 1 (A1.4) 

(iii) If A + B is the union of two mutually exclusive events in the class %, then 

P(A + B) = P(A) + P(B) (A1.5) 

Properties (i), (ii), and (iii) are known as the axioms of probability . Axiom (i) states that 
the probability of the sure event is unity. Axiom (ii) states that the probability of an event 

Sample point 

_ . /— . . • 

1 2 3 4 5 6 



Figure Al.l 



One-dimensional sample space 
Sample space for the experiment of throwing a die. 
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is a nonnegative real number that is less than or equal to unity. Axiom (iii) states that the 
probability of the union of two mutually exclusive events is the sum of the probabilities 
of the individual events. These three axioms are sufficient to deal with experiments with 
finite sample spaces. 

Although the axiomatic approach to probability theory is abstract in nature, all three 
axioms have relative-frequency interpretations of their own. Axiom (ii) corresponds to 
Equation (Al.l). Axiom (i) corresponds to the limiting case of Equation (Al.l) when the 
event A occurs in all the n trials. To interpret axiom (iii), we note that if event A occurs 
N„(A) times in n trials and event B occurs N„(B) times, then the union event "A or B" 
occurs in N„(A) + N„(B) trials (since A and B can never occur on the same trial). Hence, 
N„(A + B) = N„(A) + N„(B), and so we have 

N„(A + B) = N„(A) N„(B) 
n n n 

which has a mathematical form similar to that of axiom (iii). 

Axioms (i), (ii), and (iii) constitute an implicit definition of probability. We may use 
these axioms to develop some other basic properties of probability, as described next. 

Property 1 

P(A) = 1 - P(A) (A1.6) 
where A (denoting "not A") is the complement of event A. 

The use of this property helps us investigate the nonoccurrence of an event. To prove 
it, we express the sample space S as the union of two mutually exclusive events A and A: 

S = A + A 

Then, the use of axioms (i) and (iii) yields 

1 = P(A) + P(A) 

from which Equation (A1.6) follows directly. 

Property 2 

IfM mutually exclusive events A t , A 2 , . . . , A M have the exhaustive property 

A, + A 2 • ■ • + A M = S (A1.7) 

then 

P(A X ) + P(A 2 ) + • • • + P(A M ) = 1 (A1.8) 

To prove this property, we first use axiom (i) in Equation (A1.7), and so write 

P(A 1 + A 2 + • • • + A M ) = 1 
Next, we generalize axiom (iii) by writing 

P(A 1 + A 2 + ■■■ + A M ) = P(A X ) + P(A 2 ) + • • • + P(A M ) 
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Hence, the result of Equation (A1.8) follows. When the M events are equally likely (i.e., 
they have equal probabilities of occurrence), then Equation (A1.8) simplifies as 

P(A,.)=^_, * = 1,2,...,M 

Property 3 

When events A and B are not mutually exclusive, then the probability of the union event 
"A or B" equals 

P(A + B) = P(A) + P(B) - P(AB) (A1.9) 
where P(AB) is the probability of the joint event "A and B." 

The probability P{AB) is called a joint probability. It has the following relative- 
frequency interpretation: 

P(AB) = 1^N^ (AU0) 

where N„(AB) denotes the number of times the events A and B occur simultaneously in n 
trials of the experiment. Axiom (iii) is a special case of Equation (A1.9); when A and B 
are mutually exclusive, P(AB) is'zero, and Equation (A1.9) reduces to the same form as 
Equation (A1.5). 



■ Conditional Probability 

Suppose we perform an experiment that involves a pair of events A and B. Let P{B \A) 
denote the probability of event B, given that event A has occurred. The probability 
P{B | A) is called the conditional probability of B given A. Assuming that A has nonzero 
probability, the conditional probability P(B \ A) is defined by 

P(B|A) = ^ (Al.ll) 

where P(AB) is the joint probability of A and B. 

We justify the definition of conditional probability given in Equation (Al.ll) by 
presenting a relative-frequency interpretation of it. Suppose that we perform an experiment 
and examine the occurrence of a pair of events A and B. Let N„(AB) denote the number 
of times the joint event AB occurs in n trials. Suppose that in the same n trials, the event 
A occurs N„(A) times. Since the joint event AB corresponds to both A and B occurring, it 
follows that N„(A) must include N„(AB). In other words, we have 

N,(AB) 

N„(A) 

The ratio N„{AB)/N„(A) represents the relative frequency of B given that A has occurred. 
For large n, this ratio equals the conditional probability P(B | A); that is, 
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or, equivalently, 

N„(AB)!n 
N„(A)ln 



N„(A) 



P{B\A) = lim 

Recognizing that 

P{AB) = lim 

and 

P(A) = lim 

the result of Equation (Al.ll) follows. 

We may rewrite Equation (Al.ll) as 

P(AB) = P(B\A)P(A) (A1.12) 

It is apparent that we may also write 

P(AB) = P(A\B)P(B) (A1.13) 

Accordingly, we may state that the joint probability of two events may be expressed as 
the product of the conditional probability of one event given the other, and the elementary 
probability of the other. Note that the conditional probabilities P(B | A ) and P(A \ B) have 
essentially the same properties as the various probabilities previously defined. 

Situations may exist where the conditional probability P(A \ B) and the probabilities 
P{A) and P(B) are easily determined directly, but the conditional probability P(B\A) is 
desired. From Equations (A1.12) and (A1.13), it follows that, provided P(A) # 0, we may 
determine P(B \ A) by using the relation 

mA) ,wm (A1 , 4) 

This relation is a special form of Bayes' rule. 

Suppose that the conditional probability P(B\A) is simply equal to the elementary 
probability of occurrence of event B, that is, 

P(B\A) = P(B) 

Under this condition, the probability of occurrence of the joint event AB is equal to the 
product of the elementary probabilities of the events A and B: 

P(AB) = P(A)P(B) 

so that 

P{A\B) = P(A) 

That is, the conditional probability of event A, assuming the occurrence of event B, is 
simply equal to the elementary probability of event A. We thus see that in this case a 
knowledge of the occurrence of one event tells us no more about the probability of oc- 
currence of the other event than we knew without that knowledge. Events A and B that 
satisfy this condition are said to be statistically independent. 
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| A1.2 Random Variables 



It is customary, particularly when using the language of sample space, to think of the 
outcome of an experiment as a variable that can wander over the set of sample points and 
whose value is determined by the experiment. A function whose domain is a sample space 
and whose range is some set of real numbers is called a random variable of the experiment. 
However, the term random variable is somewhat confusing. First, the word random is not 
used in the sense of equal probability of occurrence, for which it should be reserved. 
Second, the word variable does not imply dependence (on the experimental outcome), 
which is an essential part of the meaning. Nevertheless, the term is so deeply imbedded in 
the literature of probability that its usage has persisted. 

When the outcome of an experiment is 5, the random variable is denoted as X(s) or 
simply X. For example, the sample space representing the outcomes of the throw of a die 
is a set of six sample points that may be taken to be the integers 1, 2, . . . , 6. Then if we 
identify the sample point k with the event that k dots show when the die is thrown, the 
function X(k) = k is a random variable such that X(k) equals the number of dots that 
show when the die is thrown. In this example, the random variable takes only a discrete 
set of values. In such a case, we say that we are dealing with a discrete random variable. 
More precisely, the random variable X can take only a finite number of values in any finite 
observation interval. If, however, the random variable X can take any value in a whole 
observation interval, X is called a continuous random variable. For example, the random 
variable that represents the amplitude of a noise voltage at a particular instant of time is 
a continuous random variable because it may take any value between plus and minus 
infinity. 

To proceed further, we need a probabilistic description of random variables that 
works equally well for discrete as well as continuous random variables. Let us consider 
the random variable X and the probability of the event X < x. We denote this probability 
by P(X =s x). It is apparent that this probability is a function of the dummy variable x. To 
simplify the notation, we write 



The function F x (x) is called the cumulative distribution function (cdf ) or simply the dis- 
tribution function of the random variable X. Note that F x {x) is a function of x, not of the 
random variable X. However, it depends on the assignment of the random variable X, 
which accounts for the use of X as subscript. For any point x, the distribution function 
F x {x) expresses a probability. 

The distribution function F x (x) has the following properties, which follow directly 
from Equation (A1.15): 

1. The distribution function F x (x) is bounded between zero and one. 

2. The distribution function F x (x) is a nondecreasing function of x; that is, 



An alternative description of the probability of the random variable X is often ui 
This is the derivative of the distribution function, as shown by 



F x (x) = F(X £ x) 



(A1.15) 



Fx(*i) =s F x (x 2 ) 



if Xi < x 2 



(A1.16) 



/*<*) = 




(A1.17) 
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which is called the probability density function (pdf ) of the random variable X. Note that 
the differentiation in Equation (A1.17) is with respect to the dummy variable x. The name, 
density function, arises from the fact that the probability of the event < X £ x 2 equals 

P( Xl <Xsx 2 ) = P(X ' x 2 ) - P(X £ Xi ) 

= F x (x 2 ) - F x (x,) (A1.18) 

fx(x) dx 

The probability of an interval is therefore the area under the probability density function 
in that interval. Putting Xl = -co in Equation (A1.18), and changing the notation some- 
what, we readily see that the distribution function is defined in terms of the probability 
density function as follows: 



Fx(x) = J__ f x [€) d£ (A1.19) 

Since F x {t*>) = 1, corresponding to the probability of a certain event, and F x {~°°) — 0, 
corresponding to the probability of an impossible event, we readily find from Equation 
(A1.18)that 

f x (x) dx = 1 (A1.20) 

Earlier we mentioned that a distribution function must always be nondecreasing. This 
means that its derivative or the probability density function must always be nonnegative. 
Accordingly, we may state that a probability density function must always be a nonneg- 
ative function, and with a total area of one. 

Thus far we have focused attention on situations involving a single random variable. 
However, we find frequently that the outcome of an experiment requires several random 
variables for its description. We now consider situations involving two random variables. 
The probabilistic description developed in this way may be readily extended to any number 
of random variables. 

Consider two random variables X and Y. We define the joint distribution function 
Fx,y{x, y) as the probability that the random variable X is less than or equal to a specified 
value x and that the random variable Y is less than or equal to a specified value y. The 
variables X and Y may be two separate one-dimensional random variables or the com- 
ponents of a single two-dimensional random variable. In either case, the joint sample space 
is the xy-plane. The joint distribution function F x ,y{x, y) is the probability that the outcome 
of an experiment will result in a sample point lying inside the quadrant (— °° < X s x, 
— =o < Y s y) of the joint sample space. That is, 

Fx,y{*> y) = P(X s a:, Y < y) (A1.21) 

Suppose that the joint distribution function F Xt y(x, y) is continuous everywhere, and 
that the partial derivative 

f^,y) = ^f^- (A1.22, 

ax ay 

exists and is continuous everywhere. We call the function / x ,y(*> y) the joint probability 
density function of the random variables X and Y. The joint distribution function 
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Fx y(x, y) is a nondecreasing function of both x and y. Therefore, from Equation (Al, 22) 
it follows that the joint probability density function f x , Y (x, y) is always nonnegative. Also 
the total volume under the graph of a joint probability density function must be unity, as 
shown by 

£J_/x,v(£ V)dfdr, = 1 (A1.23) 

The probability density function for a single random variable (X, say) can be ob- 
tained from its joint probability density function with a second random variable (Y, say) 
in the following way. We first note that 

F x (x) = j_ K / x ,y(£ V) d£ d v (A1.24) 

Therefore, differentiating both sides of Equation (A1.24) with respect to x, we get the 
desired relation: 

fx(x) = j_ /x,y(*. V) dv (A1.25) 

Thus the probability density function f x {x) is obtained from the joint probability density 
function / x> Y (x, y) by simply integrating it over all possible values of the undesired random 
variable Y. The use of similar arguments in the other dimension yields / Y (y). The proba- 
bility density functions f x (x) and f Y (y) are called marginal densities. Hence, the joint 
probability density function f x , Y (x, y) contains all the possible information about the joint 
random variables X and Y. 

Suppose that X and Y are two continuous random variables with joint probability 
density function f x , Y {x, y). The conditional probability density function of Y given that 
X = x is defined by 

My\*) = fj j^T (A1 - 26) 

provided that f x (x) > 0, where f x (x) is the marginal density of X. The function f Y {y \x) 
may be thought of as a function of the variable y, with the variable x arbitrary, but fixed. 
Accordingly, it satisfies all the requirements of an ordinary probability density function, 
as shown by 

f Y (y\x) a 0 

and 

\_J Y (y\x)dy = 1 

If the random variables X and Y are statistically independent, then knowledge of the 
outcome of X can in no way affect the distribution of Y. The result is that the conditional 
probability density function f Y (y\x) reduces to the marginal density f Y (y), as shown by 

f Y (y\x) = My) 

In such a case, we may express the joint probability density function of the random vari- 
ables X and Y as the product of their respective marginal densities, as shown by 

/x,y(*> v) = fx(x)fr(y) 
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In words, we may state that if the joint probability density function of the random variables 
X and Y equals the product of their marginal densities, then X and Y are statistically 
independent. 



1 A 1.3 Statistical Averages 

Having discussed probability and some of its ramifications, we now seek ways for deter- 
mining the average behavior of the outcomes arising in random experiments. 
The expected value or mean of a random variable X is defined by 

fix = E[X] = xf x (x) dx (A1.27) 

where E denotes the statistical expectation operator. That is, the mean fi x locates the center 
of gravity of the area under the probability density curve of the random variable X. To 
interpret the expected value fi x , we write the integral in the defining Equation (A1.27) as 
the limit of an approximating sum formulated as follows. Let [x k \k = 0, ±1, ±2, . . .} 
denote a set of uniformly spaced points on the real line: 

xk = (k + A, k = 0, ±1, ±2, . . . (A1.28) 

where A is the spacing between adjacent points. We may then rewrite Equation (A1.27) 
as the limiting form of a sum: 

r{k + l) A 

E[X] = lim 2 x k f x (x)dx 

A-»0 k — ~ J k A 

, ^ / A A\ (A1.29) 

= l™ Z x k P\ x k --<X^x t +-\ 

For a physical interpretation of the sum on the right-hand side of Equation (A1.29), sup- 
pose that we make n independent observations of the random variable X. Let N n (k) denote 
the number of times that the random variable X falls inside the kth bin: 

A A 
Xk-j<X&x k +-, k = 0, ±1, ±2,... 

Then, as the number of observations, », is made large, the ratio N„(k)/n approaches the 
probability P{x k - A/2 <X£j; 1 + A/2). Accordingly, we may approximate the expected 
value of the random variable X as 



k— » \ » 



l 

= ~ 2 x kN n {k), n large 



(A1.30) 



We now recognize the quantity on the right-hand side of Equation (A1.30) simply as the 
sample average. The sum is taken over all the values x k , each of which is weighted by the 
number of times it occurs; the sum is then divided by the total number of observations to 
give the sample average. Indeed, Equation (A1.30) provides the basis for computing the 
expected value E[X], 
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We next consider a more general situation. Let X denote a random variable, and let 
g{X) denote a function of X defined on the real line. The quantity obtained by letting the 
argument of the function g(X) be a random variable is also a random variable, which we 
denote as 

Y = g(X) (A1.31) 

To find the expected value of the random variable Y, we could of course find the probability 
density function f Y (y) and then apply the standard formula 

E[Y] = yf Y (y) dy 

A simpler procedure, however, is to write 

E\g(X)\ = \"__ g(x)f x (x) dx (A1.32) 

Indeed, Equation (A1.32) may be viewed as generalizing the concept of expected value to 
an arbitrary function g(X) of a random variable X. 



m Moments 

For the special case of g(X) = X", using Equation (A1.32) we obtain the nth moment of 
the probability distribution of the random variable X; that is, 

£[X»] = J__ x"f x (x) dx (A1.33) 

By far the most important moments of X are the first two moments. Thus putting n = 1 
in Equation (A1.33) gives the mean of the random variable as shown in Eq. (A1.27), 
whereas putting n = 2 gives the mean-square value of X: 

£[X 2 ] = f x 2 f x (x) dx (A1.34) 

J — x 

We may also define central moments, which are simply the moments of the difference 
between a random variable X and its mean fi x . Thus, the nth central moment is 

E[(X - Mx)"] = (* - ^x)"fx(x) dx (A1.35) 

For n = 1, the central moment is, of course, zero, whereas for n = 2 the second central 
moment is referred to as the variance of the random variable X, which is written as 

var[X] = £[(X - n x f] = {__ (* " Mx)7xM dx (A1.36) 

The variance of a random variable X is commonly denoted as The square root of the 
variance, namely, <r x , is called the standard deviation of the random variable X. 

The variance (A of a random variable X in some sense is a measure of the variable s 
"randomness." By specifying the variance ajc, we essentially constrain the effective width 
of the probability density function f x (x) of the random variable X about the mean M*- 
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A precise statement of this constraint is due to Chebyshev. The Chebyshev inequality states 
that for any positive number e, we have 

P(|X- (A1.37) 

From this inequality we see that the mean and variance of a random variable give a partial 
description of its probability distribution, hence their common use in practice. 

We note from Equations (A1.34) and (A1.36) that the variance Ox and mean-square 
value £[X 2 ] are related by 

<A = E[X 2 - 2 Ax X + & 

= E[X 2 ] - 2/x x £[X] + A (A1.38) 
= E[X 2 ] - & 

where, in the second line, we have used the linearity property of the statistical expectation 
operator E. Equation (A1.38) shows that if the mean /i x is zero, then the variance Ox and 
the mean-square value E[X 2 ] of the random variable X are equal. 



■ Characteristic Function 

Another important statistical average is the characteristic function <j> x (v) of the probability 
distribution of the random variable X, which is defined as the expectation of the complex 
exponential function exp(/fX), as shown by 

<t>x(v) = E[exp(jvX)] 

r (A1.39) 

where v is real and / = V—T. In other words, the characteristic function <j> x (v ) is (except 
for a sign change in the exponent) the Fourier transform of the probability density function 
fx(x); the Fourier transform is reviewed in Appendix 2. In this relation we have used 
exp(jvx) rather than exp(— jvx), so as to conform with the convention adopted in proba- 
bility theory. Recognizing that v and x play analogous roles to the variables 2ir/ and t of 
Fourier transforms, respectively, we deduce the following inverse relation from analogy 
with the inverse Fourier transform: 

i n 

fx(x) = — 4>x(v) exp(-/fx) dv (A1.40) 

This relation may be used to evaluate the probability density function f x {x) of the random 
variable X from its characteristic function 4>x{v). 



m Joint Moments 

Consider next a pair of random variables X and Y. A set of statistical averages of impor- 
tance in this case is the joint moments, namely, the expected value of X'Y k , where i and 
k may assume any positive integer values. We may thus write 



£[X'Y*] = J__ |__ x'y k fxMx, J) dx dy 



(A1.41) 
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A joint moment of particular importance is the correlation deifned by E[XY], which cor- 
responds to i = k = 1 in Equation (A1.41), 

The correlation of the centered random variables X - £[X] and Y - E[Y\, that is 
the joint moment 

cov[XY] = £[(X - E[X])(Y - £[Y])] (A1.42) 

is called the covariance of X and Y. Letting fj, x = £[X] and /i y = £[Y], we may expand 
Equation (A1.42) to obtain the result 

cov[XY] = E[XY] - MxA <, Y (A1.43) 

Let Ox and "y denote the variances of X and Y, respectively. Then the covariance of X 
and Y, normalized with respect to o- x o- Y , is called the correlation coefficient of X and Y: 

cov[XY] 

P = " ~ A1.44) 

°~X°~Y 

We say that the two random variables X and Y are uncorrelated if and only if their 
covariance is zero, that is, if and only if 

cov[XY] = 0 

We say that they are orthogonal if and only if their correlation is zero, that is, if and 
only if 

E[XY] = 0 

From Equation (A1.43) we observe that if one of the random variables X and Y or both 
have zero means, and if they are orthogonal, then they are uncorrelated, and vice versa. 
Note also that if X and Y are statistically independent, then they are uncorrelated; however, 
the converse of this statement is not necessarily true. 



Representation of 
Signals and Systems 



A2 . 1 Fourier Analysis 



Let g(t) denote a nonperiodic deterministic signal, expressed as some function of time t. 
By definition, the Fourier transform of the signal g(t) is given by the integral 



where / = V— 1, and the variable / denotes frequency. Given the Fourier transform G(f), 
the original signal g(t) is recovered exactly using the formula for the inverse Fourier 
transform: 



Note that in Equations (A2.1) and (A2.2) we have used a lowercase letter to denote the 
time function and an uppercase letter to denote the corresponding frequency function. The 
functions g(f) and G(f) are said to constitute a Fourier-transform pair. 

For the Fourier transform of a signal g(t) to exist, it is sufficient but not necessary 
that ,£>(£) satisfies three conditions known collectively as Dirichlet's conditions: 

1. The function g(t) is single-valued, with a finite number of maxima and minima in 
any finite time interval. 

2. The function g(t) has a finite number of discontinuities in any finite time interval. 

3. The function g(t) is absolutely integrable, that is, 



We may safely ignore the question of the existence of the Fourier transform of a time 
function g(t) when it is an accurately specified description of a physically realizable signal. 
In other words, physical realizability is a sufficient condition for the existence of a Fourier 
transform. Indeed, we may go one step further and state that all energy signals, that is, 
signals g{t) for which 



are Fourier transformable. 

The Fourier transform provides the mathematical tool for measuring the frequency 
content, or spectrum, of a signal. For this reason, the terms Fourier transform and spectrum 
are often used interchangeably. Thus, given a signal g(t ) with Fourier transform G(f), we 
may refer to G(f) as the spectrum of the signal g(t). By the same token, we refer to | G(/)| 
as the magnitude spectrum of the signal g(t), and refer to arg {G(/)} as its phase spectrum. 




(A2.1) 




(A2.2) 
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a Properties of the Fourier Transform 

It is useful to have insight into the relationship between a time function g(t) and its Fourier 
transform G(f), and also into the effects that various operations on the function g(t) have 
on the transform G{f). This may be achieved by examining certain properties of the Fourier 
transform, which are summarized in Table A6.2. 



■ Dirac Delta Function 

Strictly speaking, the theory of the Fourier transform is applicable only to time functions 
that satisfy the Dirichlet conditions. Such functions include energy signals. However, it 
would be highly desirable to extend this theory in two ways: 

1. To combine the Fourier series and Fourier transform into a unified theory, so that 
the Fourier series may be treated as a special case of the Fourier transform. 

2. To include power signals (i.e., signals for which the average power is finite) in the 
list of signals to which we may apply the Fourier transform. 

It turns out that both of these objectives can be met through the "proper use" of the Dirac 
delta function, or unit impulse. 

The Dirac delta function or just delta function, denoted by S(t), is denned as having 
zero amplitude everywhere except at t = 0, where it is infinitely large in such a way that 
it contains unit area under its curve; that is, 

S(t) = 0, t * 0 (A2.3) 

and 

f_ 8(f) dt = 1 (A2.4) 

An implication of this pair of relations is that the delta function 5(f) must be an even 
function of time f, which is centered at t = 0. 

For the delta function to have meaning, however, it has to appear as a factor in the 
integrand of an integral with respect to time and then, strictly speaking, only when the 
other factor in the integrand is a continuous function of time. Let g(t) be such a function, 
and consider the product of g{t) and the time-shifted delta function S(t - f 0 ). In light of 
the two defining equations, Equations (A2.3) and (A2.4), we may express the integral of 
this product as follows: 

J g(f) S(f - to) dt = g(t 0 ) (A2.5) 

The operation indicated on the left-hand side of this equation sifts out the value g(t 0 ) of 
the function g(t) at time t = t 0 , where -°° < t < =°. Accordingly, Equation (A2.5) is 
referred to as the sifting property of the delta function. This property is sometimes used 
as the defining equation of a delta function; in effect, it incorporates Equations (A2.3) and 
(A2.4) into a single relation. 

Noting that the delta function 5(f) is an even function of f, we may rewrite Equation 
(A2.5) so as to emphasize its resemblance to the convolution integral, as shown by 

f g(r) S(t -r)dr = g(t) (A2.0 
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In words, the convolution of any function with the delta function leaves that function 
unchanged. We refer to this statement as the replication property of the delta function. 

It is important to realize that no function in the ordinary sense has the two properties 
of Equations (A2.3) and (A2.4) or the equivalent sifting property of Equation (A2.5). 
However, we can imagine a sequence of functions that have progressively taller and thinner 
peaks at t = 0, with the area under the curve remaining equal to unity, whereas the value 
of the function tends to zero at every point except t = 0, where it tends to infinity. That 
is, we may view the delta function as the limiting form of a pulse of unit area as the 
duration of the pulse approaches zero. It is immaterial what sort of pulse shape is used. 



a Fourier Transforms of Periodic Signals 

It is well known that by using the Fourier series, a periodic signal can be represented as a 
sum of complex exponentials. Also, in a limiting sense, Fourier transforms can be defined 
by complex exponentials. Therefore, it seems reasonable to represent a periodic signal in 
terms of a Fourier transform, provided that this transform is permitted to include delta 
functions. 

Consider then a periodic signal g T „{t ) of period T 0 . We can represent g To {t ) in terms 
of the complex exponential Fourier series: 

gr 0 (t) = S c„ exp(/2i7«/ 0 f) (A2.7) 

n— — =o 

where c„ is the complex Fourier coefficient defined by 
1 f T "' 2 

c„ = — g To (t) exp(-/2ir»/„) dt (A2.8) 

and /o is the fundamental frequency defined as the reciprocal of the period T 0 ; that is, 

fo = ^r (A2.9) 

Let g(t) be a pulselike function, which equals gT 0 (t) over one period and is zero elsewhere; 
that is 

(/ , T 0 ^ ^ _ T 0 

grjt), 2 2 (A2.10) 

0 elsewhere 

The periodic signal g T<) (t ) may now be expressed in terms of the function g(t) as an infinite 
summation, as shown by 

ST„(t) = 2 g(t~ mTo) (A2.ll) 

Based on this representation, we may view g(t) as a generating function, which generates 
the periodic signal g Ta (t)- 

The function g(t) is Fourier transformable. Accordingly, we may rewrite the formula 
for the complex Fourier coefficient as follows: 

c„ = fo \_ g(t) exp(-/2ir«/ 0 f) dt 

= f 0 G(nf 0 ) (A2.12) 
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where G(nf 0 ) is the Fourier transform of git) evaluated at the frequency nf 0 . We may thus 
rewrite the formula for the reconstruction of the periodic signal g To {t) as 

gr 0 (f) = /o E G(«/ 0 ) expijlTrnfot) (A2.13) 

or, equivalently, in light of Equation (A2.ll) 

X g(t - mT 0 ) = /, | G(»/ 0 ) expijlTrnfot) (A2.14) 

Equation (A2.14) is one form of Poisson's sum formula. 

It is of interest to observe that the function git), which constitutes one period of the 
periodic signal g T it), has a continuous spectrum defined by G(f). On the other hand, the 
period signal g T (f ) itself has a discrete spectrum. We conclude, therefore, that periodicity 
in the time domain has the effect of changing the frequency-domain description or spec- 
trum of the signal into a discrete form defined at integer multiples of the fundamental 
frequency. 

m Fourier-Transform Pairs 

Table A6.3 presents a listing of some commonly used Fourier-transform pairs, the deri- 
vations of which follow from the material just presented. 

■ Transmission of Signals Through Linear Systems 

A system refers to any physical device that produces an output signal in response to an 
input signal. It is customary to refer to the input signal as the excitation and to the output 
signal as the response. In a linear system, the principle of superposition holds; that is, the 
response of a linear system to a number of excitations applied simultaneously is equal to 
the sum of the responses of the system when each excitation is applied individually. 

In the time domain, a linear system is described in terms of its impulse response, 
which is defined as the response of the system (with zero initial conditions) to a unit impulse 
or delta function Sit) applied to the input of the system. If the system is time invariant, 
then the shape of the impulse response is the same no matter when the unit impulse is 
applied to the system. Thus, assuming that the unit impulse or delta function is applied at 
time t = 0, we may denote the impulse response of a linear time-mvanant system by hit). 
Let this system be subjected to an arbitrary excitation x(t). The response, y(t), of the system 
is defined in terms of the impulse response h(t) by 

yit) = |_ x(r)h{t - r) dr (A2.15) 

which is called the convolution integral. Equivalently, we may write 

y ( t ) = I h(r) xit - t) dr (A2.16) 

Hence, convolution is commutative. 

In the convolution integral, three different time scales are involved: excitation time 
t response time t, and system-memory time t - r. This relation is the basis of time-domain 
analysis of linear time-invariant systems. According to Equation (A2.15), the present value 
of the response of a linear time-invariant system is a weighted integral over the past history 
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of the input signal, weighted according to the impulse response of the system. Thus the 
impulse response acts as a memory function for the system. 



■ Frequency Response of Linear Time-Invariant Systems 

Consider a linear time-invariant system of impulse response h{t) driven by a complex 
exponential input of unit amplitude and frequency /, that is, 

x(t) = exp(/'27r/f) 

Using this excitation in Equation (A2.16), the response of the system is obtained as 

y(t) = |_ h(r) exp[j2TTf{t - r)] dr 

(" (A2.17) 
= exp(j2Trft) J ^ h(r) exp(-/2ir/r) dr 

Define the frequency response of the system as the Fourier transform of its impulse re- 
sponse, as shown by 

H(/) = j h(t) exp(-/27r/V) dt (A2.18) 

The integral in the last line of Equation (A2.17) is the same as that of Equation (A2.18), 
except that ris used in place of t. Hence, we may rewrite Equation (A2.17) in the form 

y(£) = H(f) expijlTrft) (A2.19) 

The response of a linear time-invariant system to a complex exponential function of fre- 
quency / is, therefore, the same complex exponential function multiplied by a constant 
coefficient H(f). 

The frequency response H(/) is, in general, a complex quantity, so we may express 
it in the form 

H(f) = |H(/)|exp[/)3(/)] (A2.20) 

where \ H(f) | is called the magnitude response, and jS(/) is the phase, or phase response. 
In the special case of a linear system with a real-valued impulse response h(t), the frequency 
response H(f) exhibits conjugate symmetry, which means that 

IW)I = |H(-/)| 

and 

P(f) = -£(-/> 

That is, the magnitude response | H(f ) | of a linear system with real-valued impulse response 
is an even function of frequency, whereas the phase )3(/) is an odd function of frequency. 

In some applications, it is preferable to work with the logarithm of H(f) expressed 
in polar form rather than with H(f) itself. Define the natural logarithm 

log H(f) = «(/) + mf) (A2.21) 

where 

a(f) = log|H(/)| (A2.22) 
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The function a(f) is called the gain of the system. It is measured in nepers, whereas 
is measured in radians. Equation (A2.21) indicates that the gain a[f) and phase /3(/) are 
the real and imaginary parts of the (natural) logarithm of the frequency response H(f\ 
respectively. The gain may also be expressed in decibels (dB) by using the definition 

a'{f) = 20log 10 |H(/)| (A2.23) 

The two gain functions a(f) and a'{f) are related by 

a' {f) = $.69a(f) (A2.24) 

That is, 1 neper is equal to 8.69 dB. 

| A2.2 Bandwidth 

The time-domain and frequency-domain descriptions of a signal are inversely related. In 
particular, we may make the following important statements: 

1. If the time-domain description of a signal is changed, the frequency-domain descrip- 
tion of the signal is changed in an inverse manner, and vice versa. This inverse re- 
lationship prevents arbitrary specifications of a signal in both domains. In other 
words, we may specify an arbitrary function of time or an arbitrary spectrum, but 
we cannot specify both of them together. 

2. If a signal is strictly limited in frequency, the time-domain description of the signal 
will trail on indefinitely, even though its amplitude may assume a progressively 
smaller value. We say a signal is strictly limited in frequency or strictly band limited 
if its Fourier transform is exactly zero outside a finite band of frequencies. The sine 
pulse 

sinc(t) = 

77T 

is an example of a strictly band-limited signal. It is also asymptotically limited in 
time, which confirms the opening statement we made for a strictly band-limited 
signal. In an inverse manner, if a signal is strictly limited in time (i.e., the signal is 
exactly zero outside a finite time interval), then the spectrum of the signal is infinite 
in extent, even though the amplitude spectrum may assume a progressively smaller 
value. This behavior is exemplified by a rectangular pulse. Accordingly, we may state 
that a signal cannot be strictly limited in both time and frequency. 

The bandwidth of a signal provides a measure of the extent of significant spectral 
content of the signal for positive frequencies. When the signal is strictly band limited, the 
bandwidth is well defined. For example, the sine pulse sinc(2Wt) has a bandwidth equal 
to W. However, when the signal is not strictly band limited, as is generally the case, we 
encounter difficulty in defining the bandwidth of the signal. The difficulty arises because 
the meaning of "significant" attached to the spectral content of the signal is mathematically 
imprecise. Consequently, there is no universally accepted definition of bandwidth. Nev- 
ertheless, there are some commonly used definitions for bandwidth, as discussed next. 

When the spectrum of a signal is symmetric with a main lobe bounded by well-defined 
nulls (i.e., frequencies at which the spectrum is zero), we may use the main lobe as the 
basis for defining the bandwidth of the signal. Specifically, if the signal is low-pass (i.e., 
its spectral content is centered around the origin), the bandwidth is defined as one half the 
total width of the main spectral lobe since only one half of this lobe lies inside the positive 
frequency region. For example, a rectangular pulse of duration T seconds has a main 
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spectral lobe of total width 2/T hertz centered at the origin. Accordingly, we may define 
the bandwidth of this rectangular pulse as 1/T hertz. If, on the other hand, the signal is 
band-pass with main spectral lobes centered around ±/„ where f c is large enough, the 
bandwidth is denned as the width of the main lobe for positive frequencies. This definition 
of bandwidth is called the null-to-null bandwidth. For example, an RF pulse of duration 
T seconds and frequency f c has main spectral lobes of width 2/T hertz centered around 
±f„ where it is assumed that is large compared to 1/T. Hence, we may define the null- 
to-null bandwidth of this RF pulse as 2/T hertz. On the basis of the definitions presented 
here, we may state that shifting the spectral content of a low-pass signal by a sufficiently 
large frequency has the effect of doubling the bandwidth of the signal; such a frequency 
translation is attained by using modulation. 

Another popular definition of bandwidth is the 3-dB bandwidth. Specifically, if the 
signal is low-pass, the 3-dB bandwidth is defined as the separation between zero frequency, 
where the magnitude spectrum attains its peak value, and the positive frequency, at which 
the amplitude spectrum drops to 1/V2 of its peak value. For example, the decaying ex- 
ponential exp(-af) has a 3-dB bandwidth of a/2ir hertz. If, on the other hand, the signal 
is band-pass, centered at ±f c , the 3-dB bandwidth is defined as the separation (along the 
positive frequency axis) between the two frequencies at which the magnitude spectrum of 
the signal drops to 1/V2 of the peak value of f c . The 3-dB bandwidth has the advantage 
in that it can be read directly from a plot of the magnitude spectrum. However, it has the 
disadvantage in that it may be misleading if the magnitude spectrum has slowly decreasing 
tails. 

Yet another measure for the bandwidth of a signal is the root mean square (rms) 
bandwidth, which is defined as the square root of the second moment of a properly nor- 
malized form of the squared magnitude spectrum of the signal about a suitably chosen 
point. We assume that the signal is low-pass, so that the second moment may be taken 
about the origin. As for the normalized form of the squared magnitude spectrum, we use 
the nonnegative function 

|G(/)| 2 



f_jG(f)\ 2 df 

in which the denominator applies the correct normalization in the sense that the integrated 
value of this ratio over the entire frequency axis is unity. We may thus formally define the 
rms bandwidth of a low-pass signal g(t) with Fourier transform G(f) as follows: 



/ 2 |G(/)| 2 ^ 



\G(f)\ 2 df 



(A2.25) 



An attractive feature of the rms bandwidth W rms is that it lends itself more readily to 
mathematical evaluation than the other two definitions of bandwidth, but it is not as easily 
measurable in the laboratory. 



■ Time-Bandwidth Product 

For any family of pulse signals that differ by a time-scaling factor, the product of the 
signal's duration and its bandwidth is always a constant, as shown by 



(duration X bandwidth) = constant 
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The product is called the time-bandwidth product or bandwidth-duration product. The 
constancy of the time-bandwidth product is another manifestation of the inverse relation- 
ship that exists between the time-domain and frequency-domain descriptions of a signal. 
In particular, if the duration of a pulse signal is decreased by reducing the time scale by a 
factor a, the frequency scale of the signal's spectrum, and therefore the bandwidth of the 
signal, is increased by the same factor a, by virtue of the time-scaling property of the Fourier 
transform, and the time-bandwidth product of the signal is thereby maintained constant; 
see item 2 of Table A6.2. For example, a rectangular pulse of duration T seconds has a 
bandwidth (defined on the basis of the positive-frequency part of the main lobe) equal to 
1/T hertz, making the time-bandwidth product of the pulse equal unity. Whatever defini- 
tion we use for the bandwidth of a signal, the time-bandwidth product remains constant 
over certain classes of pulse signals. The choice of a particular definition for bandwidth 
merely changes the value of the constant. 

To be more specific, consider the rms bandwidth defined in Equation (A2.25). The 
corresponding definition for the rms duration of the signal g(t) is 




(A2.26) 



where it is assumed that the signal g(t) is centered around the origin. It may be shown 
that, using the rms definitions of Equations (A2.25) and (A2.26), the time-bandwidth 
product has the following form: 

T W (A2.27) 

A rms " rms — A v ' 

where the constant is l/4ir. The Gaussian pulse exp(-irt 2 ) satisfies this condition with the 
equality sign. 

m Noise Equivalent Bandwidth 

The definitions of bandwidth just presented (i.e., 3-dB bandwidth, null-to-null bandwidth, 
and rms bandwidth) are all formulated in terms of deterministic signals. Another definition 
of bandwidth that presents itself in the study of random signals and systems is the noise 
equivalent bandwidth. Suppose that a white noise source of power spectral density N 0 /2 
is connected to the input of the simple RC low-pass filter of Figure A2.1; the corresponding 
value of the average output noise power is equal to N 0 /(4RC). For this filter, the half- 
power or 3-dB bandwidth is equal to 1/(2 irRC). Here again we find that the average output 
noise power of the filter is proportional to the bandwidth. 

We may generalize this statement to include all kinds of low-pass filters by defining 
a noise equivalent bandwidth as follows. Suppose that we have a source of white noise of 
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Figure A2.1 RC low-pass filter. 
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Figure A2.2 Illustrating the definition of noise-equivalent bandwidth for a low-pass filter. 



zero mean and power spectral density N 0 /2 connected to the input of an arbitrary low- 
pass filter of transfer function H(f). The resulting average output noise power is therefore 



(A2.28) 



f_jH(f)\ 2 df 

r o [ IW)| 2 <f/ 
Jo 



= N< 

where, in the last line, we have made use of the fact that the magnitude response | H(f) | 
is an even function of frequency. 

Consider next the same source of white noise connected to the input of an ideal low- 
pass filter of zero-frequency response H(0) and bandwidth B. In this case, the average 
output noise power is 

N out = N 0 BH 2 (0) (A2.29) 

Therefore, equating this average output noise power to that in Equation (A2.28), we may 
formally define the noise equivalent bandwidth as 



, \W)\ 2 df 

B ' ° fito) (A2 ' 29) 

Thus the procedure for calculating the noise equivalent bandwidth consists of replacing 
the arbitrary low-pass filter of transfer function H(f) by an equivalent ideal low-pass filter 
of zero frequency response H(0) and bandwidth B, as illustrated in Figure A2.2. In a similar 
way, we may define a noise equivalent bandwidth for bandpass filters. 



A2.3 Hilhert Transform 



The Fourier transform is particularly useful for evaluating the frequency content of an 
energy signal or, in a limiting sense, that of a power signal. As such, it provides the math- 
ematical basis for analyzing and designing frequency-selective filters for the separation of 
signals on the basis of their frequency content. Another method of separating signals is 
based on phase selectivity, which uses phase shifts between the pertinent signals to achieve 
the desired separation. The simplest phase shift is that of 180 degrees, which is merely a 
polarity reversal in the case of a sinusoidal signal. Shifting the phase angles of all com- 
ponents of a given signal by 180 degrees requires the use of an ideal transformer. Another 
phase shift of interest is that of ±90 degrees. In particular, when the phase angles of all 
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components of a given signal are shifted by ±90 degrees, the resulting function of time is 
known as the Hilbert transform of the signal. 

To be specific, consider a signal g{t) with Fourier transform G(f). The Hilbert trans- 
form of g(t), which we shall denote by g{t), is defined by 

«<>=iE^* (A2 - 3i > 

Clearly, the Hilbert transformation of g{t) is a linear operation. The inverse Hilbert trans- 
form, by means of which the original signal g(t) is recovered from g(t), is denned by 

g (t) = f i^- dr (A2.32) 

77 ~=o t T 

The functions g(f) and g(f) are said to constitute a Hilbert-transform pair. A short table 
of Hilbert-transform pairs is given in Table A6.4. 

We note from the definition of the Hilbert transform that g(t) may be interpreted as 
the convolution of g(t) with the time function II m. We also know from the convolution 
theorem that the convolution of two functions in the time domain is transformed into the 
multiplication of their Fourier transforms in the frequency domain; see item 12 of Table 
A6.2. For the time function l/irt, we have (see Table A6.3) 

^---/sgn(/) (A2.33) 

where sgn(/) is the signum function defined in the frequency domain as 

f 1, />0 

sgn(/) = | 0, 7 = 0 (A2.34) 

It follows therefore that the Fourier transform G(f) of g(t) is given by 

G(/) = -/ sgn(/)G(/) (A2.35) 
Equation (A2.35) states that given a signal g{t), we may obtain its Hilbert transform 
g{t) by passing g(t) through a linear two-port device whose frequency response is equal to 
-;' sgn(/). This device may be considered as one that produces a phase shift of -90 degrees 
for all positive frequencies of the input signal and +90 degrees for all negative frequencies, 
as in Figure A2.3. The amplitudes of all frequency components in the signal, however, are 
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FIGURE A2.3 Phase characteristic of linear two-port device for obtaining the Hilbert transform 
of a real-valued signal. 



A2.4 Complex Representation of Signals and Systems 72 5 

unaffected by transmission through the device. Such an ideal device is referred to as a 
Hilbert transformer. 

■ Properties of the Hilbert Transform 

The Hilbert transform differs from the Fourier transform in that it operates exclusively in 
the time domain. It has a number of useful properties, some of which are listed next. The 
signal g(t ) is assumed to be real valued, which is the usual domain of application of the 
Hilbert transform. For this class of signals, we may state the following: 

1. A signal g(t) and its Hilbert transform g(t) have the same magnitude spectrum. 

2. If g(t) is the Hilbert transform of g{t), then the Hilbert transform of g(t) is ~g(t). 

3. A signal g(t) and its Hilbert transform g(f ) are orthogonal over the entire time interval 
(-co, co) ( as shown by 

|__ g(t)g(t)dt = 0 

Proofs of these properties are left as exercises for the reader; the proofs follow from Equa- 
tions (A2.31), (A2.32) and (A2.35). 
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m Pre-Envelope 

Consider a real-valued signal g(t). We define the pre-envelope, or analytic signal, of the 
signal g{t ) as the complex-valued function 

g4t) = g(t) + ig(t) (A2.36) 

where g{t) is the Hilbert transform of g(t). We note that the given signal g(t) is the real 
part of the pre-envelope g+(t), and the Hilbert transform of the signal is the imaginary 
part of the pre-envelope. Just as the use of phasors simplifies manipulations of alternating 
currents and voltages, so we find that the pre-envelope is particularly useful in handling 
band-pass signals and systems. 

One of the important features of the pre-envelope g+(t) is the behavior of its Fourier- 
transform. Let G+(/) denote the Fourier transform of g+(t). Then we may write 

G + (/) = G{f) + sgn(/)G(/) 

from which we readily find that 

-2G(/), / > 0 
G + (/) = | G(0), / = 0 (A2.37) 
.0, / < 0 

where G(0) is the value of G(/) at frequency / = 0. This means that the pre-envelope of 
a signal has no frequency content (i.e., its Fourier transform vanishes) for all negative 
frequencies. 
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From the foregoing analysis it is apparent that for a given signal g(t ) we may deter- 
mine its pre-envelope g+{t) in one of two equivalent ways: 

1. We determine the Hilbert transform g(t) of the signal g(t), and then use Equation 
(A2.36) to compute the pre-envelope g+(t ). 

2. We determine the Fourier transform G(/) of the signal g{t), use Equation (A2.37) to 
determine G + [f ), and then evaluate the inverse Fourier transform of G + (/) to obtain 

g + (t) = 2 I G(f) txp(jlTTft) df (A2.38) 

For a particular signal g(t) of Fourier transform G(f), one of these two ways may be better 
than the other. 

Equation (A2.36) defines the pre-envelope g + (t) for positive frequencies. Symmetri- 
cally, we may define the pre-envelope for negative frequencies as 

g-(t) = g(t) ~ >g(t) (A2.39) 
The two pre-envelopes g + (t) and g-(t) are simply the complex conjugate of each other, as 
shown by 

g-it) = g'At) (A2.40) 

where the asterisk denotes complex conjugation. The spectrum of the pre-envelope g + (t) 
is nonzero only for positive frequencies, as emphasized in Equation (A2.37); hence, the 
use of a plus sign as the subscript. In contrast, the spectrum of the other pre-envelope g_(f) 
is nonzero only for negative frequencies, as shown by the Fourier transform 

TO, / > 0 
G _(/) = J G(0), / = 0 (A2.41) 

Ug(/), / < 0 

Thus the pre-envelopes g + (f ) and g_(t) constitute a complementary pair of complex-valued 
signals. Note also that the sum of g + (f) and g-(t) is exactly twice the original signal g(t). 

a Canonical Representations of Band-Pass Signals 

Consider a band-pass signal g(t) whose Fourier transform G(/) is nonnegligible only in a 
band of frequencies of total extent 2W, say, centered about some frequency ±f c . This is 
illustrated in Figure KlAa. We refer to f c as the carrier frequency. In the majority of 
communication signals, we find that the bandwidth 2W is small compared with /„ and 
so we refer to such a signal as a narrowband signal. However, a precise statement about 
how small the bandwidth must be for the signal to be considered narrowband is not 
necessary for our present discussion. 

Let the pre-envelope of a narrowband signal g(t), with its Fourier transform M/) 
centered about some frequency ±f c , be expressed in the form 

g + (t) = g(t)txp(j2TrfJ) (A2.42) 
We refer to g{t) as the complex envelope of the signal. Equation (A2.42) may be viewed 
as the basis of a definition for the complex envelope g(t ) in terms of the pre-envelope 
We note that the spectrum of g+(f) is limited to the frequency band f c -W^fSf c +W, 
as illustrated in Figure A2.4&. Therefore, applying the frequency-shifting property ot tne 
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Figure A2.4 (a) Magnitude spectrum of band-pass signal g(t). (b) Magnitude spectrum of pre- 
envelope g. ; (t). (c) Magnitude spectrum of complex envelope g(t). 



Fourier transform to Equation (A2.42), which is described as item 5 in Table A6.2, we 
find that the spectrum of the complex envelope g(t) is limited to the band - W ■& f £ W 
and centered at the origin as illustrated in Figure A2.4c. That is, the complex envelope 
g(t) of a band-pass signal g(t) is a low-pass signal, which is an important result. 

By definition, the given signal g(t) is the real part of the pre-envelope g+(t). We may 
thus express the original band-pass signal g(t) in terms of the complex envelope g(t) as 
follows: 

g(t) = Reg(r) exp(/277£f)] (A2.43) 

In general, g(t) is a complex-valued quantity; to emphasize this property, we may express 
it in the form 



g(t) = gi(t) + )g a (t) 



(A2.44) 
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where gj(t) and g Q (t) are both real-valued low-pass functions; their low-pass property is 
inherited from the complex envelope g (f). We may therefore use Equations (A2.43) and 
(A2.44) to express the original band-pass signal g(t) in the canonical, or standard, form: 

g(t) = gl (t) cos(2ir£f) - g Q (t) sin(2irf c f) (A2.45) 

We refer to gj{t) as the in-phase component of the band-pass signal g(t) and to g Q (t) as 
the quadrature component of the signal; this nomenclature recognizes that sin(2 nf c t) [i.e., 
the multiplying factor of g Q (t)] is in phase-quadrature with respect to cos(2ir£f) [i.e., the 
multiplying factor of &(f)] and cos(2-n/ c t) is viewed as the reference. 

According to Equation (A2.44), the complex envelope g{t) may be pictured as a time- 
varying phasor positioned at the origin of the {g„ g Q )-plane, as indicated in Figure A2.5a. 
With time t varying, the end of the phasor moves about in the plane. Figure A2.56 shows 
the phasor representation of the complex exponential exp(/2 irfj). In the definition given 
in Equation (A2.43), the complex envelope g(t) is multiplied by the complex exponential 
exp(/2irf e «). The angles of these two phasors therefore add and their lengths multiply, as 
shown in Figure A2.5c. Moreover, in this latter figure, we show the (g h gg)-plane rotating 
with an angular velocity equal to 2irf c radians per second. Thus, in the picture portrayed 
here, the phasor representing the complex envelope g(t) moves in the (g„ g Q )-plane and 
at the same time the plane itself rotates about the origin. The original band-pass signal 
g{t) is the projection of this time-varying phasor on a fixed line representing the real axis, 
as indicated in Figure A2.5c. 
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FIGURE A2.5 Illustrating an interpretation of the complex envelope g(t) and its multiplicatii 
by exp(j2ir/ c i). 
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Since both gj(t) and gQ(t) are low-pass signals limited to the band — W s f < W, 
they may be derived from the band-pass signal g(t) using the scheme shown in Figure 
A2.6a. Both low-pass filters in this figure are identical, each of which has a bandwidth 
equal to W. To reconstruct g(t) from its in-phase and quadrature components, we may 
use the scheme shown in Figure A2.6b. 

The two schemes shown in Figure A2.6 are basic to the study of linear modulation 
systems. The multiplication of the low-pass in-phase component gj(t) by cos(2ir£f) and 
the multiplication of the low-pass quadrature component gQ(t) by sin(2irf c t) represent 
linear forms of modulation. Given that the carrier frequency f c is sufficiently large, the 
resulting band-pass function g(t) defined in Equation (A2.45) is referred to as a passband 
signaling waveform. Correspondingly, the mapping from gj(f) andg Q (f) intog(() is known 
as passband modulation. 

Equation (A2.44) is the Cartesian form of expressing the complex envelope g ((). 
Alternatively, we may express it in the polar form 

g(t) = a(t) «ap[#(t)] (A2.46) 

where a(t) and 4> (f ) are both real-valued low-pass functions. Based on this polar represen- 
tation, the original band-pass signal g(t) is defined by 

g(t) = a(t) cos[2tt/ c ( + 4>(t)] (A2.47) 

We refer to a(t ) as the natural envelope or simply the envelope of the band-pass signal g(t) 
and to 4>(t) as the phase of the signal. Equation (A2.47) represents a hybrid form of 
amplitude modulation and angle modulation; indeed, it includes amplitude modulation, 
frequency modulation, and phase modulation as special cases. 

From this discussion it is apparent that, whether we represent a band-pass (modu- 
lated) signal g(t) in terms of its in-phase and quadrature components as in Equation 
(A2.45) or in terms of its envelope and phase as in Equation (A2.47), the information 
content of the signal g(t) is completely preserved in the complex envelope g(t). 
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Figure A2.6 (a) Scheme for deriving the in-phase and quadrature components of a band-pass 
signal, (b) Scheme for reconstructing the band-pass signal from its in-phase and quadrature 
components. 
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■ Terminology 

The distinctions among the three different envelopes that we have introduced to describe 
a band-pass signal g(t) should be carefully noted. We summarize their definitions here: 

1. The pre-envelope g+(t) for positive frequencies is defined by 

g + (t) = g{t) + jg(t) 

where g(t) is the Hilbert transform of the signal g(t). According to this representation, 
g(t) may be viewed as the quadrature function of g(t). Correspondingly, in the fre- 
quency domain we have 

(2G(f), f > 0 
G + (/)= G(0), f = 0 
k / < 0 

2. The complex envelope g(t) equals a frequency-shifted version of the pre-envelope 
g+(t), as shown by 

g(t) = g + (t) exp(-/2i7/ c t) 

where f c is the carrier frequency of the band-pass signal g(t). 

3. The envelope a(t) equals the .magnitude of the complex envelope g(t) and also that 
of the pre-envelope g+(t), as shown by 

a(t) = | |(f) I = 

Note that for a band-pass signal g(t), the pre-envelope g + (t) is a complex band-pass signal 
whose value depends on the carrier frequency f c . On the other hand, the envelope a{t) is 
always a real low-pass signal and, in general, the complex envelope f (f ) is a complex low- 
pass signal; the values of the latter two envelopes are independent of the choice of the 
carrier frequency f c . This property gives the complex envelope g(t) an analytic advantage 
over the original signal g(t). 

The envelope a(t) and phase 4>(t) of g(t) are related to the quadrature components 
g,(t) and g Q {t) as follows (see the time-varying phasor representation of Figure A2.Sa): 

a(t) = Vgftt) + tftf 



m = tan 



Conversely, we may write 



g,{t) = a(t) cos[rf>(«)] 
g Q {t) = a(t) sin[4>(t)\ 

Thus, each of the quadrature components of a band-pass signal contains both amplitude 
and phase information. Both components are required for a unique definition of the phase 
4>{t), modulo 2tt. 



■ Band-Pass Systems 

Now that we know how to handle the complex low-pass representation of band-pass 
signals, it is logical that we develop a corresponding procedure for handling the analysis 
of band-pass systems. Specifically, we wish to show that the analysis of band-pass systems 
can be greatly simplified by establishing an analogy (or, more precisely, an isomorphism) 
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between low-pass and band-pass systems. This analogy is based on the use of the Hilbert 
transform for the representation of band-pass signals. 

Consider a narrowband signal x(t), with its Fourier transform denoted by X{f). We 
assume that the spectrum of the signal x(t) is hrnited to frequencies within ± WHz of the 
carrier frequency f c . Also, we assume that W < f c . Let this signal be represented in terms 
of its in-phase and quadrature components as follows: 

x(t) = xAt) cos(2irf c t) - x e {t) sm(2irf c t) (A2.48) 

where Xi(t) is the in-phase component and x a (t) is the quadrature component. Then, using 
x(t) to denote the complex envelope of x(t), we may write 

x(t) = x,(t) + jx Q (t) (A2.49) 

Let the signal x(t ) be applied to a linear time-invariant band-pass system with impulse 
response b{t) and frequency response H(f). We assume that the frequency response of the 
system is limited to frequencies within ±B of the carrier frequency f c . The system band- 
width IB is usually narrower than or equal to the input signal bandwidth 2 W. We wish 
to represent the band-pass impulse response b(t) in terms of two quadrature components, 
denoted by h^t) and h Q (t). Thus, by analogy to the representation of band-pass signals, 
we may express h(t) in the form 

h(t) = h,(t) cos(27r/ e i) - h Q {t) sin(2<ir/ e J) (A2.50) 

Define the complex impulse response of the band-pass system as 

h(t) = h,(t) + jb B (t) (A2.51) 

Hence, we have the complex representation 

h(t) = Re[Mf) exp(;2ir£f)] (A2.52) 

Note that hj(t), hg(t), and h(t) are all low-pass functions hmited to the frequency band 
-Bs/sB. 

We may determine the complex impulse response h(t) in terms of the quadrature 
components hj(t) and bg(t) of the band-pass impulse response h(t) by using Equation 
(A2.51). Alternatively, we may determine it from the band-pass frequency response H(f) 
in the following way. We first note from Equation (A2.52) that 

2h{t) = h(t) exp(j2irf c t) + h*(t) exp(-j2irf c t) (A2.53) 

where h*(t) is the complex conjugate of h(t). Therefore, applying the Fourier transform 
to Equation (A2.53), and using the complex-conjugation property of the Fourier trans- 
form, which is described in item 10 in Table A6.2, we get 

2H(f) = H(f - f c ) + H*(-f - f c ) (A2.54) 

where H(f) is the Fourier transform of h(t), and H(f) is the Fourier transform of h(t). 
Equation (A2.54) satisfies the requirement that H*[f) = H(—f) for a real impulse response 
h(t). Since H(f) represents a low-pass frequency response limited to | / s B with B < f c , 
we deduce from Equation (A2.54) that 

H(f - f c ) = 2H(f), f > 0 (A2.55) 

Equation (A2.55) indicates that for a specified band-pass frequency response H(/), we 
may determine H(/) by taking the part of H(f) corresponding to positive frequencies, 
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shifting it to the origin and then scaling it by the factor 2. To_ determine the complex 
impulse response h(t), we take the inverse Fourier transform of H(f), obtaining 



h(t) 



J__H(/) exp(/27r/t) df 



(A2.56) 



The representations just described for band-pass signals and systems provide the basis 
of an efficient method for determining the output of a band-pass system driven by a band- 
pass signal. We assume that the spectrum of the input signal x(t) and the frequency re- 
sponse H(f) of the system are both centered around the same frequency f. In practice, 
there is no need to consider a situation in which the carrier frequency of the input signal 
is not aligned with the midband frequency of the band-pass system, since we have consid- 
erable freedom in choosing the carrier or midband frequency. Thus, changing the carrier 
frequency of the input signal by an amount A/„ say, simply corresponds to absorbing (or 
removing) the factor exp(±/2-n- Aft) in the complex envelope of the input signal or the 
complex impulse response of the band-pass system. We are therefore justified in proceeding 
on the assumption that X(/) and H(f) are both centered around f c . Suppose then we use 
y(t) to denote the output signal of the system. It is clear that y(t) is also a band-pass signal, 
so that we may represent it in terms of its low-pass complex envelope y(t), as follows: 



y(t) = Ke[y(t) exp(/2ir/ e f)] 



(A2.57) 



The output signal y(t) is related to the input signal x{t) and impulse response h(t) of 
the system in the usual way by the convolution integral 



y(t) 



T) X (t 



(A2.58) 



In terms of pre- envelopes, we have h(t) = Re[M*)] andx(t) = Ke[x+(t)]. We may therefore 
rewrite Equation (A2.58) in terms of the pre-envelopes x+(t) and h+(t) as follows: 



y(t) 



Re[fc + (r)] Re[x + (f - t)] dr 



(A2.59) 



To proceed further, we make use of a basic property of pre-envelopes that is described by 
the following relation (presented here without proof): 



Ke[h + (r)] Re[x + (r)] dr = - Re 



j h+(T)x+(T) dr 



(A2.60) 



where we have used t as the integration variable to be consistent with that in Equation 
(A2.59). Next, we note that using x(-t) in place of x(t) has the effect of removing the 
complex conjugation on the right-hand side of Equation (A2.60). Hence, bearing in mind 
the algebraic difference between the argument of x+{r) in Equation (A2.60) and that of 
x + (t - t) in Equation (A2.59), and using the relationship between the pre-envelope and 
complex envelope of a band-pass function, we get 



y(t) 



-jRe 



Re 



i dr 



j_ h + (T) X + (t 

j_ h(r) exp(/2i7-/ e T)x(f - t) exp(/2i7/L(f - t)) dr 
exp(/2ir/ c f) j h(r)x{t - t) dr 



(A2.61) 
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Thus comparing the right-hand sides of Equations (A2.57) and (A2.61), we readily deduce 
that for a large enough carrier frequency f c , the complex envelope y(t) of the output signal 
is related to the complex envelope x(t) of the input signal and the complex impulse response 
h(t) of the band-pass system as follows: 

2y{t) = J_ h(r)x(t - r) dr (A2.62) 

or, using the shorthand notation for convolution, 

2y{t) = h{t) ★ x(t) (A2.63) 

where -k denotes convolution. In other words, except for the scaling factor 2, the complex 
envelope y(t) of the outputsignal of a band-pass system is obtained by convolving the 
complex impulse response h(t) of the system with the complex envelope x(t) of the input 
band-pass signal. Equation (A2.63) is the result of the isomorphism, for convolution, 
between a band-pass function and the corresponding low-pass function. 

The significance of this result is that in dealing with band-pass signals and systems, 
we need only concern ourselves with the low-pass functions x(t), y(t), and h(t), representing 
the excitation, the response, and the system, respectively. That is, the analysis of a band- 
pass system, which is complicated by the presence of the multiplying factor exp( j2-7rf c t), 
is replaced by an equivalent but much simpler low-pass analysis that completely retains 
the essence of the filtering process. This procedure is illustrated schematically in Figure 
A2.7. 

The complex envelope x(t) of the input band-pass signal and the complex impulse 
response h(t) of the band-pass system are defined in terms of their respective in-phase and 
quadrature components by Equations (A2.49) and (A2.51), respectively. Substituting these 
relations in Equation (A2.63), we get 

2y(t) = [h,(t) + jh Q (t)] * [ Xl (t) + jx Q (t)] (A2.64) 

Because convolution is distributive, we may rewrite Equation (A2.64) in the equivalent 
form 

2y(t) = [hdt) * x t (t) - h Q (t) * x Q (t)] + j[h Q (t) * *,(() + *, * x Q (t)] (A2.65) 

Let the complex envelope y(t) of the response be defined in terms of its in-phase and 
quadrature components as 

y(t) = yAt) + jy Q (t) (A2.66) 

Comparing the real and imaginary parts in Equations (A2.65) and (A2.66), we have for 
the in-phase component yi(t) the relation 

2y r (t) = hj(t) * Xl (t) - h Q (t) ★ x Q (t) (A2.67) 



x(t) = Re EM exp <J2wf c t)J 




yf,t) = Re [J(ri exp {j2irf c t)\ 
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Figure A2.7 (a) Narrowband filter of impulse response h(t) with narrowband input signal x(t), 
(b) Equivalent low-pass filter of complex impulse response h(t) with complex low-pass input x(t). 



734 Appendix 2 ■ Representation of Signals and Systems 




2y,W 



2y e (f) 



FIGURE A2.8 Block diagram illustrating the relationships between the in-phase and quadrature 
components of the response of a band-pass filter and those of the input signal. 



and for the quadrature component y Q (t) the relation 

2y Q (t) = h Q (t) ★ x,{t) + h t (t) ★ x Q (t) (A2.68) 
Thus, for the purpose of evaluating the in-phase and quadrature components of the com- 
plex envelope y(t) of the system output, we may use the low-pass equivalent model shown 
in Figure A2.8. All the signals and impulse responses shown in this model are real-valued 
low-pass functions. Accordingly, this equivalent model provides a practical basis for the 
efficient simulation of band-pass filters or communication channels on a digital computer. 

To sum up, the procedure for evaluating the response of a band-pass system (with 
mid-band frequency f c ) to an input band-pass signal (of carrier frequency f c ) is as follows: 

1. The input band-pass signal x(t) is replaced by its complex envelope x (t), which is 
related to x(t) by 

x(t) = Re[x(r) exp(/2ir/ c f)] 

2. The band-pass system, with impulse response h(t), is replaced by a low-pass analog, 
which is characterized by a complex impulse response b(t) related to h(t) by 

h(t) = Ke[h(t) exp(/2ir/ c c)] 

3. The complex envelope y(t) of the output band-pass signal y(t) is obtained by con- 
volving h(t) with x(t), as shown by 

2y(t) = h(t) ★ x{t) 

4. The desired output y(t) is finally derived from the complex envelope y(t) by using 
the relation 

y(t) = ReTiW exp(/2ir/ e t)] 



Bessel Functions 

A3 . 1 Series Solution qfBessel's Equation 

In its most basic form, Bessel's equation of order n is written as 

x 2 ^\ + x^- + (x 2 - n 2 )y = 0 (A3.1) 
dx dx 

which is one of the most important of all variable-coefficient differentia] equations. 1 For 
each n, a solution of this equation is denned by the power series 

(-irff*) 

/.(*) = i „ ■ ., (A3.2) 

m= o m\(n + m)\ 

The function J„(x) is called a Bessel function of the first kind of order n. Equation (A3.1) 
has two coefficient functions, namely, llx and (1 — n 2 /x 2 ). Hence, it has no finite singular 
points except the origin. It follows therefore that the series expansion of Equation (A3.2) 
converges for all x > 0. Equation (A3.2) may thus be used to numerically calculate J„(x) 
for n = 0, 1, 2, . . . . Table A6.5 presents values of JJx) for different orders n and varying 
x. It is of interest to note that the graphs of J 0 {x) and Ji(x) resemble the graphs of cos x 
and sin x, respectively; see the graphs of Figure 2.23 in Chapter 2. 

The function ]„(x) may also be expressed in the form of an integral as 

i r 

Ux) = - cos(x sin 8 - n8) dd (A3.3) 

TV JO 

/»(*) = T~ exp(jx sine - jnd) d8 (A3.4) 

27T J-it 



or, equivalently, 



| A3. 2 Properties of the Bessel Function 

The Bessel function J„(x) has the following properties: 

1. /.(*) = (-DV-W < A3 -5) 

To prove this relation, we replace 8 by (it - 8) in Equation (A3. 3). Then, noting that 
sin(7r — 8) = sin 8, we get 

1 r 

JJx) = — cos(x sin 8 + n8 — nir) dd 
17 Jo 

= — | [cos(rt7r) cos(x sin 8 + nff) + sin(ni7) sin(x sin 8 + n8)] dd 

17 JO 
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For integer values of «, we have 

cosfnir) = (-1)" 
sin(«77) = 0 

Therefore, 

(-D" r 

J„{x) = cos(x sin e + nffj d$ (A3.6) 

77 -JO ' 

From Equation (A3. 3), we also find that by replacing « with — n: 

1 r 

]-„(x) = - cos(x sin 8 + nd) dd (A3.7) 

The desired result follows immediately from Equations (A3. 6) and (A3. 7). 

2- /„(*) = (-lfU-x) (A3.8) 

This relation is obtained by replacing x with — x in Equation (A3. 3), and then using 
Equation (A.3.6). 

3- 7-i(*)+/- + iW = ^7„(x) (A3.9) 

This recurrence formula is useful in constructing tables of Bessel coefficients; its der- 
ivation follows from the power series of Equation (A3.2). 

4. For small values of x, we have 

/.(*) - ^ (A3.10) 

This relation is obtained simply by retaining the first term in the power series of 
Equation (A3.2) and ignoring the higher-order terms. Thus, when x is small, we have 

Mx) = 1 

/i(*)=-§' (A3.ll) 
]„(x) = 0 for n > 1 

5. For large values of x, we have 

/.(*)- fTcosf*-^- ^) (A3.12) 

y TTX \ 4 2 / 

This shows that for large values of x, the Bessel function J n (x) behaves like a sine 
wave with progressively decreasing amplitude. 

6. With x real and fixed, J„(x) approaches zero as the order n goes to infinity. 

7. 2 /»(*) exp(/«$) = exp(pc sin (A3. 13) 

To prove this property, consider the sum^ln— . /»(*) exp(jn4>) and use the formula 
of Equation (A3 .4) for ]„(x) to obtain 

1 " f" 
2 /„(*) exp(/«$) = — 2 exp(/'«<^>) exp( jx sine - jnd) d9 
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Interchanging the order of integration and summation: 

E /„(*) exp(;»4>) = — ddexp(jx sin 0) 2 exp[/w(</> - 0)] (A3.14) 

» °> ZTTJ—rr „--« 

We now invoke the following relation from Fourier transform theory: 

1 °° 

5 (4>) = T" 2 exp[/»(</>)], -)7<0<t (A3.15) 

where is a delta function. Therefore, using Equation (A3. 15) in (A3. 14) and 
then applying the sifting property of the delta function, we get 

2 /»(*) exp(/»<£) = |_ expl/'x sin0) S(</> - 0) i0 
= exp(/x sin </>) 

which is the desired result. 

J) } 2 „(x) = 1 for all x (A3.16) 

To prove this property, we may proceed as follows. We observe that J„(x) is real. 
Hence, multiplying Equation (A3.4) by its own complex conjugate and summing 
over all possible values of n, we get 

2 /«(*) = 2 /_ j_ exp(jx sin d - jnd ~ jx sin (j> + jn<l>) dd d(j) 

Interchanging the order of double integration and summation: 

""f r r . (A3.17) 



(2ir) 



; dO d(j) exp[/x(sin 0 - sin</>)] X exp[/«(</> - 0)] 

J —Tr J —it „ 



Using Equation (A3. 15) in (A3. 17) and then applying the sifting property of the delta 
function, we finally get 



i /iw = 7- r is = i 



2tt- 

which is the desired result. 

Many of these properties of the Bessel function ]„{%) may also be illustrated in nu- 
merical terms by referring to Table A6.5. 

! A3.3 Modified Bessel Function 

The modified Bessel equation of order n is written as 

, d 2 y dy , 
x 2 + x - (x 1 + n 2 )y = 0 (A3.18) 
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With;' 2 = — 1, where; is the square root of —1, we may rewrite this equation as 

x 2 P^ + x^ + (fx 2 - n 2 )y = 0 
dx ax 

From this rewrite it is evident that Equation (A3.18) is nothing but Bessel's equation, 
namely, Equation (A3.1), with x replaced by jx. Thus replacing x byjx in Equation (A3.2), 
we get 

- ( - ir (fP 
/„(/*) = 2 



_o m\(n + m)\ 

I X \ 



m\(n + m)\ 

Next we note that /„( jx) multiplied by a constant will still be a solution of Bessel's equa- 
tion. Accordingly, we multiply /„( jx) by the constant /"*, obtaining 

^ \ n+lm 

This new function is called the modified Bessel function of the first kind of order n, denoted 
by I„(x). We may thus formally express a solution of the modified Bessel equation, Equa- 
tion (A3. 18), as 



i.<*) = rum 

1 



(A3. 19) 



=o m\(n + m)\ 



The modified Bessel function I„(x) is a monotonically increasing real function of the ar- 
gument x for all n, as shown in Figure A3.1 for n = 0, 1, 2. 




Figure A3.1 Modified Bessel function l n {x) of varying order n. 
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The modified Bessel function I„(x) is identical to the original Bessel function ]„{x) 
except for an important difference: The terms in the series expansion of Equation (A3.19) 
are all positive, whereas they alternate in sign in the series expansion of Equation (A3.2). 
The relationship between J„(x) and I„(x) is analogous to the way in which the trigonometric 
functions cos x and sin x are related to the hyperbolic functions cosh x and sinh x. 

An interesting property of the modified Bessel function I„{x) is derived from Equation 
(A3. 13). Specifically, replacing x by jx and the angle 4> by 8 — irll in this equation, and 
then invoking the definition of I„(x) in the first line of Equation (A3. 19), we obtain 

2 h(x) exp(jnd) = exp(x cos 6) (A3.20) 

n=~ no 

From this relation it follows that 

i r 

Ux) = t- exp(x cos 6) cos(«0) dO (A3.21) 

2tT J -tt 

This integral formula for I n (x) may, of course, also be derived from Equation (A3.4) by 
making the appropriate changes. 

When the argument x is small, we obtain the following asymptotic estimates directly 
from the series representation of Equation (A3.19): 

I 0 (x) 1 for x -» 0 (A3.22) 

and 

I„(x) -» 0 for n > 1 and x -» 0 (A3.23) 

For large values of x we have the following asymptotic estimate for I„{x), which is valid 
for all integers »id: 

I„(x) = for x -> oo (A3.24) 

Note that this asymptotic behavior of I„(x) is independent of the order n for large values 
of x. 



Notes and References 



1. Equation (A3.1) is named for the German mathematician and astronomer FriedrichWilhelm 
Bessel (1784-1846). For detailed treatments of the solution to this equation and related 
issues, see Wylie and Barrett (1982) and Watson (1966). 



Confluent 

Hypergeometric 

Functions 



A4 . 1 Hummer's Equation 

The confluent hypergeometric function 1 is a solution of Kummer's differential equation: 
d z y , dy 

where, in general, the parameters a and b are complex numbers. For the case when 
b # 0, — 1, —2, . . . , the solution of Kummer's equation is defined by the series 

, . . ax a(a + 1) x 2 
, Ha ,b-, x) = l +l - + ^J- ) - + ... (A4.2, 

where i-F^tf; b; x) denotes a confluent hypergeometric function parameterized by a and b. 
In this notation, the first subscript denotes the number of factorials in the numerator of 
the general term in Equation (A4.2), the second subscript denotes the number of factorials, 
apart from «!, in the denominator. In Equation (A4.2), both subscripts are clearly 1. 



1 A4.2 Properties of the Confluent 
Hypergeometric Function 

Property 1 

For small values of x, the confluent hypergeometric function approximates as 

Jiia; b; x) - 1 + ^ x for x 0 (A4.3) 
This property follows directly from the series expansion of Equation (A4.2). 
Property 2 

For a = — 1 and b = 1 we have the exact identity: 

ifi(-l; 1; x) = 1 - x for all x (A4.4) 
This property also follows directly from the series expansion of Equation (A4.2). 
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Property 3 

The confluent hypergeometric function for a = - 1/2 and b = 1 is related exactly to the 
modified Bessel function for all x as follows: 




(A4.5) 



where l„(x) is the modified Bessel function of order n. 

A special case of Equation (A4.5) occurs when x is large. From the definition of the 
modified Bessel function given in Appendix 3, we have the following asymptotic formula 
for large x: 

I„(x) = JL — for x -»■ oo ( A4 . 6 ) 
v2ttx 

Hence, combining Equations (A4.5) and (A4.6), we obtain the simple result 




(A4.7) 



Notes and References 



1. For a discussion of confluent hypergeometric functions, see Jeffreys and Jeffreys (1956). 
Tabulated values of these functions are presented in Abramowitz and Stegun (1965). 



Cryptography 



Secrecy is certainly important to the security or integrity of information transmission. 
Indeed, the need for secure communications is more profound than ever, recognizing that 
the conduct of much of our commerce, business, and personal matters is being carried out 
today through the medium of computers, which has replaced the traditional medium of 
papers. 

Cryptology is the umbrella term used to describe the science of secret communica- 
tions; it is derived from the Greek kryptos and logos which mean "hidden" and "word," 
respectively. 1 The subject matter of cryptology may be partitioned neatly into cryptogra- 
phy and cryptanalysis. Cryptography deals with the transformations of a message into 
coded form by encryption and the recovery of the original message by decryption. The 
original message to be encrypted (enciphered) is called the plaintext, and the result pro- 
duced by encryption is called a cryptogram or ciphertext; the latter two terms are used 
interchangeably. The set of data transformations used to do the encryption is called a 
cipher; normally, the transformations are parameterized by one or more keys. Cryptanal- 
ysis, on the other hand, deals with how to undo cryptographic communications by break- 
ing a cipher or forging coded signals that may be accepted as genuine. 

Cryptographic systems offer three important services: 

1. Secrecy, which refers to the denial of access to information by unauthorized users. 

2. Authenticity, which refers to the validation of the source of a message. 

3. Integrity, which refers to the assurance that a message was not modified by accidental 
or deliberate means in transit. 

A conventional cryptographic system relies on the use of a single piece of private and 
necessarily secret information known as the key; hence, conventional cryptography is re- 
ferred to as single-key cryptography or secret-key cryptography. 2 This form of cryptog- 
raphy operates on the premise that the key is known to the encrypter (sender) and by the 
decrypter (receiver) but to no others; the assumption is that once the message is encrypted, 
it is (probably) impossible to do the decryption without knowledge of the key. 

Public-key cryptography, 3 also called two-key cryptography, differs from conven- 
tional cryptography in that there is no longer a single secret key shared by two users. 
Rather, each user is provided with key material of one's own, and the key material is 
divided into two portions: a public component and a private component. The public com- 
ponent generates a public transformation, and the private component generates a private 
transformation. But, of course, the private transformation must be kept secret for secure 
communication between the two users. 



A5 . 1 Secret-Key Cryptography 

Basically, the flow of information in a secret-key cryptographic system is as shown in 
Figure A5.1. The message source generates a plaintext message, which is encrypted into a 
cryptogram at the transmitting end of the system. The cryptogram is sent to an authorized 
user at the receiving end over an "insecure" channel; a channel is considered insecure if 
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Figure A5.1 Block diagram of secret-key cryptographic system. 



its security is inadequate for the needs of its users. It is assumed that in the course of 
transmission the cryptogram may be intercepted by an enemy cryptanalyst* (i.e., would- 
be intruder into a cryptographic system). The requirement is to do the encryption in such 
a way that the enemy is prevented from learning the contents of the plaintext message. 

In abstract terms, a cryptographic system or cipher (for short) is defined as a set of 
invertible transformations of the plaintext space (i.e., the set of possible plaintext messages) 
into the cryptogram space (i.e., the set of all possible cryptograms). Each particular trans- 
formation corresponds to encryption (enciphering) of a plaintext with a particular key. 
The invertibility of the transformation means that unique decryption (deciphering) of the 
cryptogram is possible when the key is known. Let X denote the plaintext message, Y 
denote the cryptogram, and Z denote the key. Let F denote the invertible transformation 
producing the cryptogram Y, as follows: 



Y = F(X, Z) = F,(X) 



(A5.1) 



The transformation is intended to make the cryptogram Y useless to the enemy. At the 
receiving end of the system, the cryptogram Y is decrypted with the inverse transformation 
F 1 to recover the original plaintext message X, as shown by 



(Y, Z) = F^Y) = Fj'ffyX)) = X 



(A5.2) 



In physical terms, the cryptographic system consists of a set of instructions, a piece 
of physical hardware, or a computer program. In any event, the system is designed to have 
the capability of encrypting the plaintext (and, of course, decrypting the resulting cryp- 
togram) in a variety of ways; the particular way chosen to do the actual encryption is 
determined by the specific key. 

The security of the system resides in the secret nature of the key, which requires that 
the key must be delivered to the receiver over a secure channel (e.g., registered mail, courier 
service) as implied in Figure A5.1. The cryptographic system depicted in this figure provides 
a solution to the secrecy problem, preventing an enemy from extracting information from 
messages transmitted over an insecure communication channel. Cryptography also pro- 
vides a solution to the authentication problem, preventing an enemy cryptanalyst from 
impersonating the message sender. In this second situation, the enemy cryptanalyst is the 
one who originates a "fraudulent" cryptogram Y' that is delivered to the receiver (decryp- 
ter), as shown in Figure A5.2. The authentic cryptogram Y is shown as a dashed input to 
the enemy cryptanalyst, indicating that the enemy produces the fraudulent cryptogram Y' 
without ever seeing the authentic one. The receiver may be able to recognize Y' as fraud- 
ulent by decrypting it with the correct key Z; hence, the line from the receiver output to 
the destination is shown dashed to suggest rejection of the fraudulent cryptogram Y' by 
the receiving user. 
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Figure A5.2 Illustrating the intrusion of an enemy cryptanalyst. 



A5 . 2 B lock and Streatn Ciphers 

Much as error-correcting codes are classified into block codes and convolutional codes, 
cryptographic systems (ciphers) may be classified into two broad classes: block ciphers and 
stream ciphers. Block ciphers operate in a purely combinatorial fashion on large blocks of 
plaintext, whereas stream ciphers process the plaintext in small pieces (i.e., characters or 

bits). , ■ ■ t 

Figure A5.3 shows the generic form of a block cipher. The plaintext (consisting of 
serial data) is divided into large blocks, each of which is usually made up of a fixed number 
of bits. Successive blocks of the plaintext are enciphered (encrypted) using the same secret 
key, otherwise independently; the resulting enciphered blocks are finally converted into 
serial form. Thus, a particular plaintext block identical to a previous such block gives rise 
to an identical ciphertext block. Specifically, each bit of a particular ciphered block is 
chosen to be a function of all the bits of the associated plaintext block and the key; the 
goal of a block cipher is to have no specific bit of the plaintext ever appear in the ciphertext 

directly. , . . 

Block ciphers operate with a fixed transformation applied to large blocks of plaintext 
data, on a block-by-block basis. In contrast, a stream cipher operates on the basis of a 
time-varying transformation applied to individual bits of the plaintext. The most popular 
stream ciphers are the so-called binary additive stream ciphers, the generic form of which 
is shown in Figure A5.4. In such a cipher, the secret key is used to control a key stream 
generator that emits a binary sequence called the key stream, whose length is much larger 
than that of the key. Let *„, y„, and z n denote the plaintext bit, ciphertext bit, and key- 
stream bit at time n, respectively. The ciphertext bits are then determined by simple mod- 
ulo-2 addition of the plaintext bits and the keystream bits, as shown by 

y„ = x„@z„, n = 1, 2, . . . , N (A5.3) 
where N is the length of the keystream. Because addition and subtraction in modulo-2 
arithmetic are exactly the same, Equation (A5.3) also implies the following relation 

x„ = y„@z„, n = 1, 2, . . . , N (A5.4) 
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Figure A5.3 Block diagram of a block cipher. 
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Figure A5.4 Binary additive stream cipher. 



We thus see that in binary additive stream ciphers, identical devices can be used to perform 
encryption and decryption, as shown in Figure A5.4. The secret key is chosen according 
to some probability distribution. To provide secure encryption, the keystream should re- 
semble a coin-tossing (i.e., completely random) sequence as closely as possible. 

Block ciphers are normally designed in such a way that a small change in an input 
block of plaintext produces a major change in the resulting output. This error propagation 
property of block ciphers is valuable in authentication in that it makes it improbable for 
an enemy cryptanalyst to modify encrypted data, unless knowledge of the key is available. 
On the other hand, a binary additive stream cipher has no error propagation; the decryp- 
tion of a distorted bit in the ciphertext affects only the corresponding bit of the resulting 
output. 

Stream ciphers are generally better suited for the secure transmission of data over 
error-prone communication channels; they are used in applications where high data rates 
are a requirement (as in secure video, for example) or when a minimal transmission delay 
is essential. 5 



■ Requirement for Secrecy 

In cryptography, a fundamental assumption is that an enemy cryptanalyst has knowledge 
of the entire mechanism used to perform encryption, except for the secret key. We may 
identify the following forms of attack that may be attempted by the enemy cryptanalyst, 
depending on the availability of additional knowledge; 

1. Ciphertext-only attack is a cryptanalytic attack in which the enemy cryptanalyst has 
access to part or all of the ciphertext. 

2. Knoivn-plaintext attack is a cryptanalytic attack in which the enemy cryptanalyst 
has knowledge of some ciphertext-plaintext pairs formed with the actual secret key. 

3. Chosen-plaintext attack is a cryptanalytic attack in which the enemy cryptanalyst is 
able to submit any chosen plaintext message and receive in return the correct 
ciphertext for the actual secret key. 

4. Chosen-ciphertext attack is a cryptanalytic attack in which the enemy cryptanalyst 
is able to choose an arbitrary ciphertext and find the correct result for its decryption. 

A ciphertext-only attack occurs frequently in practice. In this form of attack, an 
enemy cryptanalyst uses only knowledge of the statistical structure of the language in use 
(e.g., in English the letter e occurs with a probability of 13 percent, and the letter q is 
always followed by u) and knowledge of some probable words (e.g., a letter probably 
begins with "Dear Sir/Madam:"). A known-plaintext attack may take place by virtue of 
the standard computer formats used in programming languages and data generation. In 
any case, the ciphertext-only attack is viewed as the weakest threat to which a crypto- 
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graphic system can be subjected, and any system that succumbs to it is therefore considered 
totally insecure. Thus, for a cryptographic system to provide secrecy, at the minimum it 
should be immune to ciphertext-only attacks; ideally, it should also be immune to known- 
plaintext attacks. 



| A5.3 Information-Theoretic Approach 

In the Shannon model of cryptography , named in recognition of Shannon's 1949 landmark 
paper on the information-theoretic approach to secrecy systems, the enemy cryptanalyst 
is assumed to have unlimited time and computing power. But the enemy is presumably 
restricted to a ciphertext-only attack. Cryptanalysis in the Shannon model is defined as the 
process of finding the secret key, given the cryptogram (ciphertext) and the a priori prob- 
abilities of the various plaintexts and keys. The secrecy of the system is considered broken 
when the enemy cryptanalyst performs decryption successfully, obtaining a unique solution 
to the cryptogram. 6 

Let X = (X 1( X 2 , . . . , X N ) denote an N-bit plaintext message, and Y = {Y u Y 2 , . . . , 
Y N ) denote the corresponding N-bit cryptogram; that is, both the plaintext and the cryp- 
togram have the same number of bits. It is assumed that the secret key Z used to construct 
the cryptogram is drawn according to some probability distribution. The uncertainty about 
X is expressed by the entropy Jf(X), and the uncertainty about X given knowledge of Y 
is expressed by the conditional entropy H(X | Y). The mutual information between X and 
Y is defined by 

I(X; Y) = H(X) - H(X\Y) (A5.5) 

The mutual information I(X; Y) represents a basic measure of security (secrecy) in the 
Shannon model. 



■ Perfect Security 

Assuming that an enemy cryptanalyst can observe only the cryptogram Y, it seems appro- 
priate that we define the perfect security of a cryptographic system to mean that the plain- 
text X and the cryptogram Y are statistically independent. In other words, we have 

I(X; Y) = 0 (AJ.6) 

Then, using Equation (A5.5), we find that the condition for perfect security may be re- 
written as 

H(X\Y) = H(X) (A5.7) 

Equation (A5.7) states that the best an enemy cryptanalyst can do, given the cryptogram 
Y, is to guess the plaintext message X according to the probability distribution of all 
possible messages. 

Given the secret key Z, we recognize that 

H(X|Y) sff(X,Z|Y) 

= H(Z\Y) + H(X\Y, Z) 

The conditional entropy H(X|Y, Z) is zero if, and only if, Y and Z together uniquely 
determine X; this is indeed a valid assumption when the decryption process is performed 
with knowledge of the secret key Z. Hence, we may simplify Equation (A5.8) as follows: 

H(X|Y)SH(Z|Y) (A59) 
sH(Z) 
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Thus, substituting Equation (A5.9) into (A5.7), we find that for a cryptographic system to 
provide perfect security, the following condition must be satisfied: 

H(Z) a H(X) (A5.10) 

The inequality of Equation (A5.10) is Shannon's fundamental bound for perfect security; 
it states that for perfect security, the uncertainty of a secret key Z must be at least as large 
as the uncertainty of the plaintext X that is concealed by the key. 

For the case when the plaintext and key alphabets are of the same size, the use of 
Shannon's bound for perfect security yields the following result: The key must be at least 
as long as the plaintext. The conclusion to be drawn from this result is that the length of 
the secret key needed to build a perfectly secure cryptographic system may be unpractically 
large for most applications. Nevertheless, perfect security has a place in the practical pic- 
ture: It may be used when the number of possible messages is small or in cases where the 
greatest importance is attached to perfect security. 

A well-known, perfectly secure cipher is the one-time pad 1 (sometimes called the 
Vernam cipher), which is used for unconventional applications such as two users com- 
municating on a hotline with high confidentiality requirements. The one-time pad is a 
stream cipher for which the key is the same as the keystream, as shown in Figure A5.5. 
For encryption the input consists of two components: a message represented by a sequence 
of message bits {x„ \ n = 1, 2, . . .}, and a key represented by a sequence of statistically 
independent and uniformly distributed bits {z n \n = 1, 2,...}. The resultant cipher 
\y n \n = 1, 2, . . .} is obtained by the modulo-2 addition of the two input sequences, as 
shown by ' 

Vn = x n © z„, n = 1, 2, . . . 

Consider, for example, the binary message sequence 00011010 and the binary key se- 
quence 01101001. The modulo-2 addition of these two sequences is written as follows: 

Message: 00011010 
Key: 01101001 
Cipher: 01110011 

In the encryption rule described here, key bit 1 interchanges 0s and Is in the message 
sequence, and key bit 0 leaves the message bits unchanged. The message sequence is re- 
covered simply by modulo-2 addition of the binary cipher and key sequences, as shown 
by 

Cipher: 01110011 

Key: 01101001 

Message: 00011010 

The one-time pad is perfectly secure, because the mutual information between the message 
and the cipher is zero; it is therefore completely undecipherable. 



Key Key 



Message a- — > Qj) s»- cipher Cipher <> — ^ >• 

Encrypter Decrypter 
Figure A5.5 One-time pad (Vernam cipher). 



748 Appendix 5 o Cryptography 



■ Unicitv Distance 

Consider now the practical case of an imperfect cipher and ask the question: When can 
an enemy cryptanalyst break the cipher? As the amount of intercepted text increases, 
intuitively we expect that a point may be reached at which it becomes possible for 
an enemy cryptanalyst with unlimited time and computing power to find the key and 
thus break the cipher. This critical point in the Shannon model is called the unicity dis- 
tance, which is formally defined as the smallest N such that the conditional entropy 
H(Z \ Yi, Y 2 , . . . , Y N ) is approximately zero. For a particular kind of "random cipher," 
the unicity distance is approximately given by 8 

No-^r- (A5.ll) 
r log L y 

where H(Z) is the entropy of the key Z, and L y is the size of the ciphertext alphabet. The 
parameter r is the percentage redundancy of the message information contained in the 
N-bit ciphertext; it is itself denned by 

where H(X) is the entropy of the plaintext X. In most cryptographic systems, the size L, 
of the ciphertext alphabet is the same as the size L x of the plaintext alphabet; in such a 
case, r is just the percentage redundancy of the plaintext itself. Although the derivation of 
Equation (A5.ll) assumes a certain well-defined "random cipher," it can be used to esti- 
mate the unicity distance for ordinary types of ciphers, which is the routine practice today. 

Let K be the number of digits in the key Z that are chosen from an alphabet of size 
L 2 ; then we may express the entropy of the key Z as follows: 

H(Z) =£ log(L?) = K log L z (A5.13) 

with equality if and only if the key is completely random. Let the size L z of the key alphabet 
be the same as the size L y of the ciphertext alphabet, and let the key be chosen completely 
at random to maximize the unicity distance. Then, substituting Equation (A5.13) with 
equality into Equation (A5.ll), we get the simple result 

N 0 = - (A5.14) 
r 

To illustrate the application of Equation (A5.14), consider a cryptographic system with 
L x = L y = L z , which is used for the encryption of English text. The percentage redundancy 
r for typical English text is about 75 percent. Hence, according to Equation (A5.14), an 
enemy cryptanalyst can break the cipher after intercepting only about 1.333K bits of 
ciphertext data, where K is the key size. 

However, it is important to note that an imperfect cipher that is potentially breakable 
can still be of practical value. When the intercepted ciphetext contains sufficient infor- 
mation to satisfy Equation (A5.ll), there is no guarantee that an enemy cryptanalyst with 
limited computational resources can actually break the cipher. Specifically, it is possible 
for the cipher to be designed in such a way that the task of the cryptanalysis, though 
known to be attainable with a finite amount of computation, is so overwhelming that it 
will literally exhaust the physical computing resources of the universe. In such a case, the 
imperfect cipher is said to be computationally secure. 
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■ Role of Data Compression in Cryptography 

Lossless data compression or data compaction is a useful tool in cryptography. We say 
this because data compaction removes redundancy, thereby increasing the unicity distance 
N 0 in accordance with Equation (A5.ll). To exploit this idea, data compaction is used 
prior to encryption in the transmitter, and the redundant information is reinserted after 
decryption in the receiver; the net result is that the authorized user at the receiver output 
sees no difference, and yet the information transmission has been made more secure. It 
would be tempting to consider the use of perfect data compaction to remove all redun- 
dancy, thereby transforming a message source into a completely random source and re- 
sulting in N 0 = °o with any key size. Unfortunately, we do not have a device capable of 
performing perfect data compaction on realistic message sources, nor is it likely that there 
will ever be such a device. It is therefore futile to rely on data compaction alone for data 
security. Nevertheless, limited data compaction tends to increase security, which is the 
reason why cryptographers view data compression as a useful trick. 

a Diffusion and Confusion 

In the Shannon model of cryptography, two methods suggest themselves as general prin- 
ciples to guide the design of practical ciphers. The methods are called diffusion and con- 
fusion, the aims of which (by themselves or together) are to frustrate a statistical analysis 
of ciphertext by the enemy and therefore make it extremely difficult to break the cipher. 

In the method of diffusion, the statistical structure of the plaintext is hidden by 
spreading out the influence of a single bit in the plaintext over a large number of bits in 
the ciphertext. This spreading has the effect of forcing the enemy to intercept a tremendous 
amount of material for the determination of the statistical structure of the plaintext, since 
the structure is evident only in many blocks, each one of which has a very small probability 
of occurrence. In the method of confusion, the data transformations are designed to com- 
plicate the determination of the way in which the statistics of the ciphertext depend on the 
statistics of the plaintext. Thus, a good cipher uses a combination of diffusion and 
confusion. 

For a cipher to be of practical value, however, it must not only be difficult to break 
the cipher by an enemy cryptanalyst, but also it should be easy to encrypt and decrypt 
data given knowledge of the secret key. We may satisfy these two design objectives using 
a product cipher, based on the notion of "divide and conquer." Specifically, the imple- 
mentation of a strong cipher is accomplished as a succession of simple component ciphers, 
each of which contributes a modest amount of diffusion and confusion to the overall 
makeup of the cipher. Product ciphers are often built using substitution ciphers and trans- 
position ciphers as basic components; these simple ciphers are described next. 

1. Substitution cipher. 

In a substitution cipher each letter of the plaintext is replaced by a fixed substitute, usually 
also a letter from the same alphabet, with the particular substitution rule being determined 
by the secret key. Thus the plaintext 

X = (x-t, x 2 , x 3 , X4, . . .) 

where x u x 2 , x 3 , . . . are the successive letters, is transformed into the ciphertext 



Y = (yi, y 2 , y 3 < y 4 , ■ ■ ■) 

= if(*l), f(Xl), f (* 3 ), fM, . . .) 



(A5.15) 
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Plaintext ABCDEFGHIJKLMNOPQRSTUVWXYZ 



letters 
Ciphertext 



YDUBHNACSVXELPFMKQJRWGOZIT 



letters 

Figure A5. 6 Substitution cipher. 



where /(•) is a function with an inverse. When the substitutes are letters, the key is a 
permutation of the alphabet. Consider, for example, the ciphertext alphabet of Figure 
A5.6, where we see that the first letter Y is the substitute for A, the second letter D is the 
substitute for B, and so on. The use of a substitution cipher results in confusion. 

2. Transposition cipher. In a transposition cipher, the plaintext is divided into groups of 
fixed period d and the same permutation is applied to each group, with the particular 
permutation rule being determined by the secret key. For example, consider the permu- 
tation rule described in Figure A5.7, for which the period is d = 4. According to this 
cipher, letter x 1 is moved from position 1 in the plaintext to position 4 in the ciphertext. 
Thus, the plaintext 

X = (Xi, X 2 , X3, X 4 , X 5 , X 7 , X$, . . .) 

is transformed into the ciphertext 

Y = (x 3 , x A , Xx, x-[, x 7 , x s , x 6 , x 5 , . . .) 

Although the single-letter statistics of the ciphertext Y are the same as those of the plaintext 
X, the higher-order statistics are changed. The use of a transposition cipher results in 
diffusion. 

By interleaving the simple substitutions and transpositions and repeating the interleaving 
process many times, it is possible to build a strong cipher equipped with good diffusion 
and confusion. 



Example A5.1 

Consider the plaintext message 

THE KING IS DEAD LONG LIVE THE KING 

Using the permuted alphabet describeiin Figure AJ.6 for the substitution cipher, this plaintext 
is transformed into the ciphertext 

RCHXSPASJBHYBEFPAESGHRCHXSPA 

Suppose next we apply the permutation rule described in Figure A5.7 for the transposition 
cipher; accordingly, the ciphertext resulting from the substitution cipher is further transformed 
into 

HXCRASPSHYBJFBEBSGEACHRHPASX 
which has no resemblance to the original plaintext. ^ 



Plaintext v 

letters Xl * 2 13 14 

Ciphertext v 
letters H H * 2 

Figure A5.7 Transposition cipher. 
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| A5.4 Data Encryption Standard 



The data encryption standard (DES)* is certainly the best known, and arguably the most 
widely used, secret-key cryptoalgorithm; the term algorithm is used to describe a sequence 
of computations. The basic DES algorithm can be used for both data encryption and data 
authentication. It is the standard cryptoalgorithm for data storage and mail systems, elec- 
tronic funds transfers (retail and wholesale), and electronic business data interchange. 

The DES algorithm is a strong block cipher that operates on 64-bit blocks of plaintext 
data and uses a 56-bit key; it is designed in accordance with Shannon's methods of dif- 
fusion and confusion. Essentially the same algorithm is used for encryption and decryption. 
The overall transformations employed in the DES algorithm may be written as 
f-'If-'lPfX)]), where X is the plaintext, P is a certain permutation, and the function F 
combines substitutions and transpositions. The function F is itself obtained by cascading 
a certain function /, with each stage of the cascade referred to as a round. 

The flow-chart of Figure A5.8 shows the details of the DES algorithm for encryption. 
After a certain initial permutation, a plaintext of 64 bits is divided into a left-half L 0 and 
a right-half B 0 , each of which is 32 bits long. The algorithm then performs 16 rounds of 
a key-dependent computation, with the rth round of the computation described as follows: 

Li= R,-i i = 1, 2, . . . , 16 (A5.16) 

R, = © /(R,_ a , Z,) i = 1, 2, . . . , 16 (A5.17) 

On the right-hand side of Equation (A5.17), the addition is modulo-2 and each Z, is a 
different 48-bit block of the key used in round i. The function /(•, •) is a function with a 
32-bit output. The result of the 16th round is reversed, obtaining the sequence R i6 L 16 . 
This 32-bit sequence is input into a final permutation P _1 to produce the 64-bit ciphertext. 
The aim is that after 16 rounds of key-dependent computations, the patterns in the original 
plaintext are undetectable in the ciphertext. From Equations (A5.16) and (A5.17), we note 
that for decryption the function /(■, -) need not be invertible, because (L.-j, R^) can be 
recovered from (L„ R ; ) simply as follows: 

Ri-i ~ i = 1, 2, ... , 16 (A5.18) 

L,_, = R, © /(L„ Z.) i = 1, 2, . . . , 16 (A5.19) 

Equation (A5.19) holds even if the function /(•, •) is a many-to-one function (i.e., it does 
not have a unique inverse). 

Figure A5.9 shows the flowchart for computing the function /(•, •). The 32-bit block 
J? is first expanded into a new 48-bit block R f by repeating the edge bits of each successive 
4-bit word (i.e., the bits numbered 1, 4, 5, 8, 9, 12, 13, 16, ... , 28, 29, 32). Thus, given 
the 32-bit block R written as 

r 29 r 30 r 31 r 32 

eighth 
4-bit word 



R = r,r 2 r,r 4 r s r 6 r 7 r a 

first second 
4-bit word 4-bit word 

we construct the expanded 48-bit block R' as follows: 



R' - W2W5 r 4 r 5 r 6 r 7 r s r 9 ■ ■ • r2ar 2 9r 30 r 31 r 32 r 1 

first second eighth 

6-bit word 6-bit word 6-bit word 




Figure A5.8 Data encryption standard. (From Diffie and Hellman, 1979, with permission of 
the IEEE.) 



A5.4 Data Encryption Standard 753 



32-Bit R 



Expansion 

m 



Permutation 
J>[ ] 



32-Bit HSZi) 



FIGURE A5.9 /(R, K) flowchart. (From Diffie and Hellman, 1979, with permission of the 
IEEE.) 



The 48-bit blocks R' and Z, are added modulo-2, and the resultant is divided into eight 
6-bit words. Let these words be denoted by B u B 2 , . . . , B s . We thus write 

B,B 2 • • • B 8 = R' © Z,- (A5.20) 

Each 6-bit word B, is input to a substitution box S, in the form of a look-up table, pro- 
ducing a 4-bit output S,(B,). Each output bit of the substitution box S, (B,) is a Boolean 
function of the 6-bit word B h The eight outputs S 1 (B 1 ), S 2 (B 2 ), . . . , S 8 (B 8 ) are arranged 
into a single 32-bit block that is input to the permutation box denoted by P| -]. The per- 
muted output so produced is the desired 32-bit function f(R, Z,), as shown by 

/(R, Z.) = PlSABJSziBi) ■ ■ ■ S S (B 8 )] (A5.21) 

The 48-bit block Z,- for the ith iteration uses a different subset of the 64-bit key Z 0 . 
The procedure used to determine each Z, is called the key-schedule calculation, the flow- 
chart of which is shown in Figure A5.10. The key Z 0 has eight parity bits in positions 8, 
16, ... , 64, which are used for error detection in their respective 8-bit bytes; the errors 



754 Appendix 5 s Cryptography 



Permuted 
choice 2 



64- Bit key 



Permuted 
choice 1 
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choice 2 



Permuted 
choice 2 



Figure A5.10 Flowchart for the key-schedule calculation. (From Diffie and Hellman, 1979, 
with permission of the IEEE.) 



of concern may arise in the generation, distribution, and storage of the key Z 0 . The per- 
muted choice 1 disregards the parity bits of Z 0 and then permutes the remaining 56 bits 
that are loaded into two 28-bit shift registers, each with 24 taps. The 48 taps of the two 
shift registers are subjected to 16 iterations of computation, with each iteration involving 
one or two cyclic left shifts followed by a permutation, referred to as permuted choice 2. 
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The outputs resulting from these 16 iterations provide the different 48-bit blocks Z u 
Z 2 , . . . , Z 16 of the key used in iteration 1, 2, ... , 16, respectively. 

Despite all the claims to the contrary, it appears that no one has yet demonstrated a 
fundamental weakness of the DES algorithm. Notwithstanding all the controversy sur- 
rounding its use, perhaps the most significant contribution of the DES algorithm is the fact 
that it has been instrumental in raising the level of interest in using cryptography as a 
mechanism for secure computer networks. 

| A5 . 5 Public-Key Cr yptography 1 0 

For a pair of users to engage in cryptographic communication over an insecure channel, 
it is necessary for the users to exchange key information prior to communication. The 
requirement for a secure distribution of keys among authorized users applies to all cryp- 
tographic systems, regardless of their type. In conventional cryptography, the users employ 
a physically secure channel (e.g., courier service or registered mail) for key distribution. 
However, the use of such a supplementary channel points to a major limitation of con- 
ventional cryptography. Needless to say, the use of courier service or registered mail for 
key distribution is costly, inconvenient, low-bandwidth, and slow; also, it is not always 
secure. 

The problem of key distribution is particularly accentuated in large communication 
networks, where the number of possible connections grows as (n 2 - n)ll for n users. For 
large n, the cost of key distribution becomes prohibitive. Thus, in the development of large, 
secure communication networks, we are compelled to rely on the use of insecure channels 
for both exchange of key information and subsequent secure communication. This con- 
straint raises a fundamental question: How can key information be exchanged securely 
over an insecure channel? In public-key cryptography, this seemingly difficult issue is re- 
solved by making some key material "public" and thereby considerably simplifying the 
task of key management. This is in direct contrast to conventional cryptography, where 
the key is kept completely secret from an enemy cryptanalyst. 

A public key cryptographic system is described by two sets of algorithms that com- 
pute invertible functions (transformations). Let these two sets of algorithms be denoted by 
[£J and {DJ that are indexed by z. The invertible transformations computed by these 
algorithms may be written as follows 

K'- AM = y (AJ.22) 
D z : f~\y) = x (AJ.23) 

where x is a certain input message in the domain of some function f z indexed by z, and y 
is the corresponding cryptogram in the range of f z . A fundamental requirement of the 
system is that the function f z must be a trapdoor one-way function. The term "one-way" 
refers to the fact that for x in the domain of f„ it must be easy to compute f z (x) from 
knowledge of the algorithm E z , but for a certain cryptogram y in the range of f z , an enemy 
cryptanalyst must find it extremely difficult to compute the inverse f^iy). On the other 
hand, an authorized user in possession of the associated algorithm D z would find it easy 
to compute the inverse f z x (y). Thus the private key (algorithm) D z provides a "trapdoor" 
that makes the problem of inverting the function f z appear extremely difficult from the 
viewpoint of the cryptanalyst, but easy for the (sole authorized) possessor of D z . Since 
knowledge of the key (algorithm) E z does not by itself make it possible to compute the 
inverse of f„ it may be made public; hence, the name "public-key cryptography." 
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The notion emerging from the description of a public-key cryptographic system pre- 
sented herein is that the keys come in inverse pairs (i.e., public key and private key), and 
that each pair of keys has two basic properties: 

1. Whatever message is encrypted with one of the keys can be decrypted with the other 
key. 

2. Given knowledge of the public key, it is computationally infeasible to find the secret 
key. 

The use of public-key cryptography as described herein makes it possible to solve 
the secrecy problem as follows. Subscribers to a secure communication system list their 
public keys in a "telephone directory" along with their names and addresses. A subscriber 
can then send a private message to another subscriber simply by looking up the public key 
of the addressee and using the key to encrypt the message. The encrypted message (i.e., 
ciphertext) can only be read by the holder of that particular public key. In fact, should the 
original message (i.e., plaintext) be lost, even its sender would find it extremely difficult to 
recover the message from the ciphertext. 

The key management of public-key cryptography makes it well suited for the devel- 
opment of large, secure communication networks. Indeed, it has evolved from a simple 
concept to a mainstay of cryptographic technology. 



■ Diffif-Hellman Public Key Distribution 

In a simple and yet elegant system known as the Diffie-Hellman public key-distribution 
system, use is made of the fact that it is easy to calculate a discrete exponential but difficult 
to calculate a discrete logarithm. To be more specific, consider the discrete exponential 
function 

Y= a x mod p for 1 < X 2= p - 1 (A5.24) 

where the arithmetic is performed modulo-p. The a is an integer that should be primitive 
(i.e., all powers of a generate all the elements mod p relatively prime to p - 1). Corre- 
spondingly, X is referred to as the discrete logarithm of Yto the base a, mod p, as shown 
by 

X = lo go Y mod p for 1 £ Y < p - 1 (A5.25) 

The calculation of Y from X is easy, using the trick of square-and-multiply. For example, 
for X = 16 we have 

Y = a" = {[(a 2 ) 2 ] 2 ) 2 

On the other hand, the problem of calculating X from Y is much more difficult. 

In the Diffie-Hellman public key-distribution system, all users are presumed to know 
both a and p. A user /', say, selects an independent random number X ; uniformly from the 
set of integers {1,2,..., p] that is kept as a private secret. But the discrete exponential 

Yi = a* mod p (A5.26) 

is deposited in a public directory with the user's name and address. Every other user of 
the system does the same thing. Now, suppose that users i and / wish to communicate 
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privately. To proceed, user i fetches Y, from the public directory and uses the private secret 
Xj to compute 

K yi = (Y ; .) x 'modp 

= (a*')*' mod p (A5.27) 
= a x ' x > mod p 

In a similar way, user /' computes K,y. But we have 

K„ = Kf (A5.28) 

Accordingly, users i and /' arrive at K ;l - as the secret key in a conventional cryptosystem. 
Another user must compute Kj, using the information Y ; and Y f obtained from the public 
directory, applying the alternative formula 

K H = (Y ; ) losY < mod p (A5.29) 

Apparently, there is no other method for an enemy to find the secret key K ;i ; however, 
there is no proof for it. In light of what we said earlier, Equation (A5.29) is difficult to 
calculate as it involves a discrete logarithm, whereas Equation (A5.27) is easy to calculate 
as it involves a discrete exponential. Thus, security of the system depends on the difficulty 
encountered in computing a discrete logarithm. 

The Diffie-Hellman public key-distribution system is the oldest system in its class; 
nevertheless, it is still generally considered to be one of the most secure and practical public 
key-distribution systems. 



| A5.6 Rivest— Shamir— Adleman System 

To develop a public-key cryptographic system is no easy task. Indeed, numerous such 
systems have been proposed in the literature, but unfortunately most of them have proven 
to be insecure. To date, the most successful implementation of public-key cryptography is 
the Rivest-Shamir-Adleman (RSA) system, 11 which uses ideas from classical number the- 
ory. It is considered to be one of the most secure cryptographic systems in that it has 
withstood many attempts by experts in the field to break it. 

The RSA algorithm is a block cipher based on the fact that finding a random prime 
number of large size (e.g., 100 digit) is computationally easy, but factoring the product of 
two such numbers is currently considered computationally infeasible. Specifically, the com- 
putation of parameters specific to the RSA algorithm proceeds as follows: 

1. Choose two very large prime numbers, p and q, at random; the prime numbers have 
to be fairly carefully chosen as some prime numbers lead to a very weak system. 

2. Multiply the numbers p and q, obtaining the product 

pq = n (A5.30) 

Find the Euler totient function of n, using the formula 

4>{n) = (p- l)(q- 1) (A5.31) 

Equation (A5.31) follows from the definition of the Euler totient function <f>(n) as 
the number of positive integers i less than n, such that the greatest common divisor 
of i and n is equal to one. 
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3. Let e be a positive integer less than 4>(n), such that the greatest common divisor of 
e and <j> (n) is equal to one. Hence, find a positive integer d less than 4> («), such that 

de = 1 mod <M«) (A5.32) 

The RSA trapdoor one-way function is then defined simply by computing the discrete 
exponentiation 

f z {x) = x' = y mod n (A5.32) 

The values of n and e constitute the public key; hence, publishing the easy-to-find algorithm 
E z to compute the function f z amounts just to publishing the numbers n and e. 

The prime numbers p and q constitute the private key. Since d is related to p and q, 
possession of the easy-to-find (when one knows the trapdoor z) algorithm D z to compute 
the inverse function / J 1 amounts just to knowing p and q. In particular, the inverse func- 
tion is denned by 

fz \y) = / mod n (A5.34) 

The decrypting exponent d is found using Equation (A5.32), which is equivalent to the 
statement (in ordinary integer arithmetic) that 

de = 4>(n)Q + 1 (A5.35) 

for some integer Q. Note that 4>(n) is itself related to p and q by Equation (A5.31). Since 
y = x", we may use Equations (A5.32) and (A5.33) to write 

y d = x de 

= x *we+i (A5.36) 
= ((x* w ) e )x 

We now make use of a celebrated theorem of Euler, which states that for any positive 
integers x and n with x < n, we have 

x* in) = 1 mod n (A5.37) 

Hence, the use of Equation (A5.37) in (A5.36) yields the desired decryption: 

■f = x (A5.38) 

We thus see that finding the inverse function /J 1 is easy, given knowledge of the prime 
numbers p and q. 

The security of the RSA cryptoalgorithm rests on the premise that any method of 
inverting the function f z is equivalent to factoring n - pq. This equivalence raises the 
question: Is an attack by factoring « computationally feasible? It appears that the answer 
is no, provided that the prime numbers p and q are on the order of 100 decimal digits 
each and that there is no revolutionary breakthrough in factoring algorithms. 



■ Digital Signatures 12 

For an electronic mail system to replace the use of ordinary paper mail for business trans- 
actions, it must be possible for a user of the system to "sign" an electronic message. The 
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use of a digital signature provides proof that the message did originate from the sender. 
To satisfy this requirement, the digital signature must have the following properties: 

The receiver of an electronic message is able to verify the sender's signature. 
»• The signature is not forgeable. 

> The sender of a signed electronic message is unable to disclaim it. 

To implement digital signatures using the RSA algorithm, we may proceed as follows. 
A user in possession of the private key d may sign a given message block m by forming 
the signature 

s = m d mod n (A5.39) 

It is difficult to compute 5 unless the private key d is known. Hence, a digital signature 
defined in accordance with Equation (A5.39) is difficult to forge. Moreover, the sender of 
message m cannot deny having sent it, since no one else could have created the signature 
s. The receiver proceeds by using the public key e to compute 

s" = (m d ) e mod n 

= m de mod n (A5.40) 
= m mod n 

where, in the last line, use is made of Equation (A5.32). Hence, the receiver is able to 
validate the sender's signature by establishing that the computation of s e mod n produces 
the same result as the deciphered message m. Thus, the RSA algorithm satisfies all the 
three necessary properties of a digital signature. 

A5.7 Summary and Discussion 

Cryptography is a "hot" research area. This statement should not come as a surprise. 
Considering the fact that we are in an information society, the importance of cryptography 
as a security mechanism will continue to grow. In this appendix, we have presented an 
introductory treatment of this highly important subject. 

We may classify cryptography into secret-key cryptography and public-key cryptog- 
raphy, depending on whether the key used for the encryption of a message and its decryp- 
tion is completely secret or partly public. Alternatively, we may classify a cryptographic 
system into a block cipher or stream cipher, depending on the method of implementation. 
A block cipher exhibits error propagation, which can prove highly valuable in 
authentication. 

Among the many cryptographic systems developed to date, the data encryption stan- 
dard (DES) and the Rivest-Shamir-Adleman (RSA) algorithms stand out as the most suc- 
cessful ones. Both of these cryptoalgorithms are block ciphers. They differ from each other 
in that the DES algorithm involves the use of a secret key whereas the RSA algorithm 
involves the use of a public key. In a secret-key system, the same key is shared both by the 
sender and the receiver. On the other hand, in a public-key system, the key is split into 
two parts: a public key located in the transmitter and a private (secret) key located in the 
receiver; in the latter system, it is computationally infeasible to recover the plaintext mes- 
sage from its encrypted version without knowledge of the private key. 

Although public-key cryptosystems such as RSA provide an effective method for key 
management, they are inefficient for the bulk encryption of data due to low bandwidths. 
In contrast, conventional cryptosystems such as DES provide better throughput, but they 
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require key management. This suggests the possible use of a hybrid approach exploiting 
the best elements of both cryptosystems as the basis for the practical design of a secure 
communication system. For example, the RSA algorithm may be used for authentication, 
and the DES algorithm for encryption. 



■ Notes and References 

1. For an introductory treatment of cryptography, see Chapter 15 of the book by Adamek 
(1991). For a comprehensive treatment of the many facets of cryptology, see the book 
edited by Simmons (1992); this book is an expanded edition of a Special Issue of the 
Proceedings of the IEEE (1988) on cryptology. The chapter contributions of the book by 
Simmons are written by leading authorities on the subject of cryptology. A nice treatment 
of cryptology is also presented in the book by van Tilborg (1988). 

2. The era of scientific secret-key cryptography was ushered in with the publication of a 
landmark paper by Shannon (1949), which established the connection between cryptog- 
raphy and information theory. 

3. The era of public-key cryptography was established with the publication of another land- 
mark paper by Diffie and Hellman (1976), which showed for the first time that it is possible 
to have secret communications without any transfer of a key between sender and receiver. 
It was the paper by Diffie and Hellman that sparked the explosion of research interest in 
cryptology, which has continued ever since. 

4. The term enemy cryptanalyst is commonly used in cryptology to refer to a cryptogram 
interceptor (eavesdropper); its usage originates from military applications. 

5. For a comprehensive treatment of stream ciphers, see Chapter 2 written by R. A. Rueppel 
in the book Contemporary Cryptology, edited by Simmons (1992). 

6. For a highly readable account of the Shannon model of cryptography, see the opening 
chapter by J. L. Massey in the book edited by Simmons (1992). 

7. The one-time pad derives its name from its use (shortly before, during, and after World 
War II) by spies of several governments, who were given a pad of paper with a randomly 
chosen key and told to use it only for a single encryption. The one-time pad is also known 
as Vernam's cipher, so named in recognition of its originator, G. S. Vernam. 

8. For a derivation of Equation (A5. 11), see the original paper by Shannon (1949). 

9. The history of the DES algorithm is recounted by M. E. Smid and D. K. Branstad in Chapter 
1 of the book edited by Simmons (1992). For a description of the DES algorithm, see Diffie 
and Hellman (1979). See also the books by Meyer and Matyas (1982) and Torrieri (1992, 
Chapter 6). 

10. For a comprehensive treatment of public-key cryptography, see Chapter 4 by J. Nechvatal 
in the book edited by Simmons (1992). This book also includes a chapter contribution by 
W. Diffie that describes the several attempts to devise secure public-key cryptoalgorithms 
and the gradual evolution of a variety of protocols based on them. 

11. The RSA system is patented; it is named in recognition of its originators R. L. Rivest, 
A. Shamir, and L. Adleman. The original reference for this cryptosystem is Rivest, Shamir, 
and Adleman (1978). 

12. The idea of a digital signature was first discussed by Diffie and Hellman (1976). Its imple- 
mentation using the RSA algorithm is described by Rivest, Shamir, and Adleman (1978). 
For a detailed treatment of digital signatures, see Chapter 6 by C. J. Michell, F. Piper, and 
R. Wild in the book edited by Simmons (1992). 



Tables 



The twelve tables compiled in this final appendix cover the following: 

► ASCII code 

► Fourier and Hilbert transforms 

► Bessel functions 

► Error function 

► Selected modem standards 

► Trigonometric identities, series expansions, and integrals 

► Useful constants and recommended unit prefixes 
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| Table A6.1 ASCII code 



Bit Position 











/ V 


U 


U 


if 


i 
i 


i 
i 


i 


1 










6 0 


o 


1 


1 


0 


0 


1 


i 
l 


4 


3 


2 


1 


S 0 


1 


0 


1 


0 


1 


0 


1 


0 


0 


0 


0 


NUL 


DLE 


SP 


0 


@ 


P 


\ 


P 


0 


0 


0 


1 


SOH 


DCl 


i 


1 


A 


Q 


a 


q 


0 


0 


1 


0 


STX 


DC2 




2 


B 


R 


b 


r 


0 


0 


1 


1 


ETX 


DC3 


# 


3 


C 


S 


c 


s 


0 


1 


0 


0 


EOT 


DC4 


$ 


4 


D 


T 


d 


t 


0 


1 


0 


1 


ENQ 


NAK 


% 


5 


E 


U 


e 


u 


0 


1 


1 


0 


ACK 


SYN 


& 


6 


F 


V 


f 


V 


0 


1 


1 


1 


BEL 


ETB 




7 


G 


w 


g 


w 




0 


0 


0 


BS 


CAN 


( 


8 


H 


X 


h 


X 




0 


0 


1 


HT 


EM 


) 


9 


I 


Y 


i 


y 




n 

u 


1 


A 

u 


T V 




» 




T 
J 




J 


Z 




0 


1 


1 


VT 


ESC 


+ 




K 


[ 


k 


f 
\ 




1 


0 


0 


FF 


FS 




< 


T 

L 


\ 


1 






1 


0 


1 


CR 


GS 






M 


] 


m 


} 




1 


1 


0 


SO 


RS 




> 


N 


A 


n 


~ 




1 


1 


1 


SI 


US 


/ 


} 


0 




o 


DEL 


ACK 


Acknowledge 




ENQ 


Enquiry- 






NUL 


Null or all 


zeros 




BEL 


Bell or 


alarm 




EOT 


End of transmission 




RS 


Record separator 




BS 


Backspace 




ESC 


Escape 






SI 


Shift in 






CAN 


Cancel 






ETB 


End of transmission block 


SO 


Shift out 






CR 


Carriage return 




ETX 


End of text 






SOH 


Start of heading 




DCl 


Device control 1 


FF 


Form feed 






SP 


Space 






DC2 


Device control 2 


FS 


File separator 






STX 


Start of text 




DC3 


Device control 3 


GS 


Group separator 




SUB 


Substitute 






DC4 


Device control 4 


HT 


Horizontal tab 






SYN 


Synchronous idle 




DEL 


Delete 






LF 


Line feed 






US 


Unit separator 




DLE 


Data link escape 


NAK 


Negative acknowledge 




VT 


Vertical tab 




EM 


End of medium 





















(From Couch, 1990, with permission of Macmilian.) 
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| Table A6.2 Summary of properties of tlte Fimrier transform 



Property 



1. Linearity 

2. Time scaling 

3. Duality 

4. Time shifting 

5. Frequency shirting 

6. Area under g(t) 

7. Area under G{f) 

8. Differentiation in the time domain 

9. Integration in the time domain 

10. Conjugate functions 

11. Multiplication in the time domain 

12. Convolution in the time domain 



Mathematical Description 



agM + bgz(t) ^ aG^f) + bG z (f) 
where a and b are constants 

where a is a constant 

If git) ^ G(f), 

then G(t)^g{-f) 

git - t 0 ) ^ G(f) ex P (-/27T/* 0 ) 

exp(/277/ e %M ^ G(f - f c ) 

\_j(t)dt = G(0) 
S(0) = J_ G(f) df 



g{t)^j2TTfG(f) 

idr^^-.G(f) + ^-d(f) 
jl-rrf 2 



dt 

L S(T - 
If J?M-G</), 
then «*(t)^G*(-/) 

gi(t)gM ^ G!(A)G 2 (/ - A) dA 
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| TABL E A6i«3 Fourier-tra n^orm pairs 

Time Function Fourier Transform 



T sinc(/T) 

J_ i± 
2W Te "\2W, 



II - 



\t\ < T 

\t\ > T 



sinc(2Wf) 

exp(-at)u(t), a> 0 

exp(-a|i|), a> 0 
exp(— Trt 2 ) 

M 

T ' 

Lo» 

S(f) 

1 

5{f - t 0 ) 
exp(;'27r/ c f) 
cos(27r£f) 



sgn(f) 
»(t) 



«T 0 ) 



a + /27r/ 
2a 

a 2 + (2ir/) 2 
exp(-ir/ 2 ) 

T siiwrVT) 

1 

S(/) 

exp(-/2ir/t 0 ) 

S(/ - fc) 

iW - ft + s(f + f c )\ 

w - fc) - «(/ + m 

i 

jirf 

-/ sgn(/) 

^ «(/) + r^-r 
2 /2ir/ 



Notes: u[t) = unit step function 

5(f) = delta function, or unit impulse 

rect(f) = rectangular function of unit amplitude and unit 

duration centered on the origin 

sgn(f) = signum function 

sinc{£) = sine function 
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■ Table A6.4 Hilbert transform, pairs" 

Time Function Hilbert Transform 



m(t) cos(2ir/ e ?) 
m(t) sin(2ir/ c i) 
cos(2irf c t) 
sin(2ir/ e *) 
sin t 
t 

rect(*) 



S(t) 
1 



1 + ? 

1 

t 



sin(2i7£r) 
-m(t) cos(2ir£*) 
5m(2Trf c t) 
~cos(2irf c t) 
1 ~ cos t 



-— log 



7rt 



1 

' + 2 



a In the first two pairs, it is assumed that m(t) is band- 
limited to the interval - W < / < W, where W < / c . 
Notes: 6(t): delta function 

rect(f): rectangular function of unit amplitude and 
unit duration centered on the origin 
log: natural logarithm 



| Table A6.5 Table of Bessel functions' 



n\x 


0.5 


1 


2 


3 


4 


6 


S 


10 


12 


0 


0.9385 


0.7652 


0.2239 


-0.2601 


-0.3971 


0.1506 


0.1717 


-0.2459 


0.0477 


1 


0.2423 


0.4401 


0.5767 


0.3391 


-0.0660 


-0.2767 


0.2346 


0.0435 


-0.2234 


2 


0.0306 


0.1149 


0.3528 


0.4861 


0.3641 


-0.2429 


-0.1130 


0.2546 


-0.0849 


3 


0.0026 


0.0196 


0.1289 


0.3091 


0.4302 


0.1148 


-0.2911 


0.0584 


0.1951 


4 


0.0002 


0.0025 


0.0340 


0.1320 


0.2811 


0.3576 


-0.1054 


-0.2196 


0.1825 


5 




0.0002 


0.0070 


0.0430 


0.1321 


0.3621 


0.1858 


-0.2341 


-0.0735 


6 






0.0012 


0.0114 


0.0491 


0.2458 


0.3376 


-0.0145 


-0.2437 


7 






0.0002 


0.0025 


0.0152 


0.1296 


0.3206 


0.2167 


-0.1703 


8 








0.0005 


0.0040 


0.0565 


0.2235 


0.3179 


0.0451 


9 








0.0001 


0.0009 


0.0212 


0.1263 


0.2919 


0.2304 


10 










0.0002 


0.0070 


0.0608 


0.2075 


0.3005 


11 












0.0020 


0.0256 


0.1231 


0.2704 


12 












0.0005 


0.0096 


0.0634 


0.1953 


13 












0.0001 


0.0033 


0.0290 


0.1201 


14 














0.0010 


0.0120 


0.0650 



Tor more extensive tables of Bessel functions, see Watson (1966, pp. 666-697), and Abramowitz and Stegun (1965. pp. 
358^106). 
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TABLE A6.6 The error function" 



u 


erf(u) 


M 


erf(u) 


0.00 


n nnnnn 

u.uuuuu 


1.10 


0.88021 


0.05 




1.15 


0.89612 


0 10 


0.11246 


1.20 


0.91031 


0.15 




1.25 


0.92290 


0.20 


0.22270 


1.30 


0.93401 


0.25 




1.35 


0.94376 


0.30 




1.40 


0.95229 


0.35 


0.37938 


1.45 


0.95970 


0.40 


0.42839 


1.50 


0.96611 


0.45 


0.47548 


1.55 


0.97162 


0.50 




1.60 


0.97635 


0.55 


U.J OJjI 


1.65 


0 98038 


0.60 


UiDUJOD 


1.70 


0.98379 


0.65 




1.75 


0.98667 


0.70 


0 67780 


1.80 


0.98909 


0.75 


0.71116 


1.85 


0.99111 


0,80 


0.74210 


1.90 


0.99279 


0.85 


0.77067 


1.95 


0.99418 


0.90 


0.79691 


2.00 


0.99532 


0.95 


0.82089 


2.50 


0.99959 


1.00 


0.84270 


3.00 


0.99998 


1.05 


0.86244 


3.30 


0.999998 



"The error function is tabulated extensively in several 
references; see for example, Abramowitz and Stegun 
(1965, pp. 297-316). 
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| TABLE A6.7 Selection of ITL ' voiceband (telephone line) modem standards 





ITU 










Ol'Urltliir u 


± yy& \jf [(ivjLimiwstivjft 


Hit r/ito h/e 


Vuw*rt/-w ¥/tto n/ltJfic 
LjyTfllSUl Tlibti) t/UKU) 


(s.) Symmetric modems; 


V.21 


Binary FSK 


300 


300 




V.22 bis 


QPSK 


1,200 


600 




V.26 


QPSK 


2,400 


1,200 




V.27 


8-PSK 


4,800 


2,400 




V.32 


16-QAM 


9,600 


2,400 




V.34 


1024-QAM 


28,800 


3,429 




V.j4 High opeed 


Nested-constellation 








of four 960-QAM 
constellations 






(b) Asymmetric modems: 


V.90: Downstream 


Digital 


56,000 






Upstream 


V.34 High Speed 


33,600 





a The suffix "bis" designates the second version of a particular standard. 



1 Table A6.8 Trigonometric identities 

exp( ±/0) = cos 0 ± j sin 0 
cos 0 = f[exp(/0) + exp(-/0)] 



: J. [exp(/fl) ■ 



exp(-/fl)] 



sin 2 0 + cos 2 0=1 

cos 2 0 - sin 2 0 = cos(20) 

cos 2 0 = j[l + cos(20)] 

sin 2 0 = J[l - cos(20)] 

2 sin 0 cos 0 = sin(20) 

sin(a ± )3) = sin a cos B ± cos a sin B 

cos(a ± /?} = cos a cos /? + sin a sin /? 

tan a ± tan B 

tan(a ± fl = — — 

1 + tan a tan B 

sin a sin /3 = §[cos(a — /3) — cos(a + B)] 

cos a cos /3 = f [cos(a — B) + cos(a + B)\ 

sin a cos B — j[sin(a — B) + s'm(a + B)] 
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| Table A6.9 Series expansions 

Taylor series 

where 



MacLaurin series 



HO) HO) , / w (0) „ , 

/(*) = /(") + ij x + 2! x ~n\ * 



where 



Binomial series 

Exponential series 

Logarithmic series 
Trigonometric series 



«(« — 1) , i i 

(1 + *)" = 1 + nx + — L x 2 + --; \nx\< 



log(l + x) = x - ix 2 + %x 3 



sin x 


1 i 1 , 




cos X 


1 , 1 4 




tan x 


1 , 2 j 


+ • • • 


sin -1 x 


1 , 3 , 
= * + 6* 3 + S* 


+ ■ • ■ 


tan 1 x 






sine x 


= 1 _1 (TO) , + I 


(17*)" " 



* < 1 
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| Table A6. 10 Integrals 

Indefinite integrals 

j x sin(ax) dx = ^ [sin(«^) — ax zos{ax)\ 

j x cos(ax) dx = -j [cos(ax) + ax sm(ax)] 

| x exp(ax) dx = exp(ax)(ax — 1) 
J a 

| x exp(ax 2 ) dx — exp(ax 2 ) 
J 2a 

| exp(ax) sm(bx) dx = -7 exp(ax)[a sin(fcx) — b cos(bx)] 

J a + b 

f 1 

J exp(tfjc) cos(fcx) lie = ^ 2 ^ 2 exp(ux)[a cos(fcx) + b sin(fcx)] 
[ dx 1 _, / bx\ 

f x 2 _ X ^ _j/ bx\ 

Definite integrals 

f " x sinlax) , it , , , 

, dx = - exp(-a6), a > 0, > 0 
Jo (r + jr 2 

Jo (r + x 2b 

[°° cos(ux) 17 . , , , , 

771 777 a* = T7"7 [sinlao) - cos(ae)], a > 0, fc > 0 

Jo (ir - x ) 46 

| sine x dx = \ sine 2 x dx = — 
Jo Jo 2 

exp(- 

Jo 



2 

1 /it 



-a**) dx = - —, a > 0 
2 V a 



exp(- 

Jo 



1 /7T 

■ax 2 ) dx = — -, a > 0 
4a V a 
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1 TABLE A6. 11 Useful constants 

I „ — 



Physical Constants 

Boltzmann's constant 
Planck's constant 
Electron (fundamental) charge 
Speed of light in vacuum 
Standard (absolute) temperature 
Thermal voltage 

Thermal energy kT at standard temperature 

One hertz (hz) = 1 cycle/second; 1 cycle = 2ir radians 

One watt (W) = 1 joule/second 

Mathematical Constants 

Base of natural logarithm 
Logarithm of e to base 2 
Logarithm of 2 to base e 
Logarithm of 2 to base 10 
Pi 



k = 1.38 X 1(T 23 joule/degree Kelvin 

h = 6.626 X KT 34 joule-second 

q = 1.602 X 10~ 19 coulomb 

c = 2.998 X 10 8 meters/second 

T 0 = 273 degrees Kelvin 

V T = 0.026 volt at room temperature 

kT 0 = 3.77 X 10~ 21 joule 



e = 2.7182818 
log 2 e = 1.442695 
log 2 = 0.693147 
log 1() 2 = 0.30103 
tt= 3.1415927 



Table A6. 1 2 Recommended unit prefixes 



Multiples and Submultiples Prefixes Symbols 

10 12 tera T 

10 9 giga G 

10 6 mega M 

10 3 kilo K (fe) 

1(T 3 milli m 

10~ 6 micro A 

10~' nano n 

10" 12 pico p 
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Conventions and Notations 



1. The symbol | | means the absolute value, or magnitude, of the complex quantity 
contained within. 

2. The symbol arg( ) means the phase angle of the complex quantity contained within. 

3. The symbol Re[ ] means the "real part of," and Im[ ] means the "imaginary part of." 

4. Unless stated otherwise, the natural logarithm is denoted by log. Logarithms to bases 
2 and 10 are denoted by log 2 and log )0 , respectively. 

5. The use of an asterisk as superscript denotes complex conjugate, e.g., x" is the com- 
plex conjugate of x. 

6. The symbol ^ indicates a Fourier-transform pair, e.g., g(t) ?± G(f), where a low- 
ercase letter denotes the time function and a corresponding uppercase letter denotes 
the frequency function. 

7. The symbol F[ ] indicates the Fourier-transform operation, e.g., F[g(t)] = G(f), 
and the symbol F _1 [ ] indicates the inverse Fourier-transform operation, e.g., 
P _1 [G(/)] = g(t). 

8. The symbol * denotes convolution, e.g., 

x(t) ★ b(t) = |_ x(r)h(t - t) dr 

9. The symbol © denotes modulo-2 addition, except in Chapter 10 where binary arith- 
metic is used and modulo-2 addition is denoted by an ordinary plus sign throughout 
that chapter. 

10. The use of subscript T 0 indicates that the pertinent function^ To (f), say, is a periodic 
function of time t with period T 0 . 

11. The use of a hat over a function indicates one of two things: 

(a) the Hilbert transform of a function, e.g., the function g(t) is the Hilbert transform 

of g(t), or 

(b) the estimate of an unknown parameter, e.g., the quantity a(x) is an estimate of 
the unknown parameter a, based on the observation vector x. 

12. The use of a tilde over a function indicates the complex envelope of a narrowband 
signal, e.g., the function g(t) is the complex envelope of the narrowband signal g{t). 
The exception to this convention is in Section 10.8, where, in the description of turbo 
decoding, the tilde is used to signify extrinsic information and thereby distinguish it 
from log-likelihood ratio. 

13. The use of subscript + indicates the pre-envelope of a signal, e.g., the function 
g+(t) is the pre-envelope of the signal g(t). We may thus write g+{t) = g(t) + jg(t), 
where g(t) is the Hilbert transform of g(t). The use of subscript - indicates that 

*-(»)= *(»)-«(») = *+•(*)• 

14. The use of subscripts I and Q indicates the in-phase and quadrature components of 
a narrowband signal, a narrowband random process, or the impulse response of a 
narrow-band filter, with respect to the carrier cos(27r/,.f). 
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15. For a low-pass message signal, the highest frequency component or message band- 
width is denoted by W. The spectrum of this signal occupies the frequency interval 
- W £ / W and is zero elsewhere. For a band-pass signal with carrier frequency 
f c , the spectrum occupies the frequency intervals, f c -Wrsfsf c +W and 
-f c - W s / < -f e + W, and so 2W denotes the bandwidth of the signal. The 
(low-pass) complex envelope of this band-pass signal has a spectrum that occupies 
the frequency interval — W £ / £ W. 

For a lowpass filter, the bandwidth is denoted by B. A common definition of filter 
bandwidth is the frequency at which the magnitude response of the filter drops by 
3 dB below the zero-frequency value. For a band-pass filter of mid-band frequency 
f c the bandwidth is denoted by 2B, centered on f c . The complex low-pass equivalent 
of this band-pass filter has a bandwidth equal to B. 

The transmission bandwidth of a communication channel, required to transmit a 
modulated wave, is denoted by B T . 

16. Random variables or random vectors are uppercase (e.g., X or X), and their sample 
values are lowercase (e.g., x or x). 

17. A vertical bar in an expression means "given that," e.g., f x (x \ H 0 ) is the probability 
density function of the random variable X, given that hypothesis H 0 is true. 

18. The symbol E[ ] means the expected value of the random variable enclosed within; 
the E acts as an operator. 

19. The symbol var[ ] means the variance of the random variable enclosed within. 

20. The symbol cov[ ] means the covariance of the two random variables enclosed 
within. 

21. The average probability of symbol error is denoted by P e . 

In the case of binary signaling techniques, p w denotes the conditional probability 
of error given that symbol 0 was transmitted, and p 01 denotes the conditional prob- 
ability of error given that symbol 1 was transmitted. The a priori probabilities of 
symbols 0 and 1 are denoted by p 0 and £>,, respectively. 

22. The symbol < > denotes the time average of the sample function enclosed within. 

23 . Boldface letter denotes a vector or matrix. The inverse of a square matrix R is denoted 
by R -1 . The transpose of a vector w is denoted by w T . The Hermitian transpose of 
a complex-valued vector x is denoted by x H ; Hermitian transposition involves both 
transposition and complex conjugation. 

24. The length of a vector x is denoted by || x ||. The Euclidean distance between the 
vectors x, and x ; is denoted by = || x,- — x ; - 1|. 

25. The inner product of two real-valued vectors x and y is denoted by x T y; their outer 
product is denoted by xy T . If the vectors x and y are complex valued, their inner 
product is x H y, and their outer product is xy". 

26. The vector product of two M-by-1 vectors a and p is an M-by-1 vector defined by 



a • P 



«2& 



where a k and ft are the kth elements of a and p, respectively. The L l norm of the 
vector product a ■ p is defined by 

M 

1<* -PlIi = S « m P™ 



Abbreviations 



I Functions 

« 

1. Rectangular function: 

2. Unit step function: 

3. Signum function: 

4. (Dirac) delta function: 

or, equivalently, 

5. Sine function: 

6. Sine integral: 

7. Error function: 
Complementary error function: 

8. Binomial coefficient 

9. Bessel function of the first kind 
of order n: 

10. Modified Bessel function of the 
first kind of zero order: 

11. Confluent hypergeometric 
function 



rectl» = 



fl, -\<t<\ 
to, \t\>\ 




sinc(x) = 



Si(«) = I m X dx 
Jo x 

2 [" 

erf(«) = —^= exp(-z 2 ) dz 
VttJo 

erfc(«) = 1 - erf(w). 



( 



(n - k)\k\ 
JJx) = T - I exp(/;c sin 8 — jnff) dd 

2.7T J— ir 

i r 

Io{x) — exp(x cos 6) dd 

ATT J-tt 

_ , . . „ ax a(a + 1) x 2 
J^b;x) = l + - b - + WTT) - 



■ Abbreviations 

I : . 



A: 


ampere 


AC: 


alternating current 


ADC: 


analog-to-digital converter 


ADM: 


adaptive delta modulation 


ADPCM: 


adaptive differential pulse-code modulation 


ADSL: 


asymmetric digital subscriber line 


AM: 


amplitude modulation 


ANSI: 


American National Standards Institute 


APB: 


adaptive prediction with backward estimation 


APF: 


adaptive prediction with forward estimation 


AQB: 


adaptive quantization with backward estimation 
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AQF: 


adaptive quantization with forward estimation 


ARQ: 


automatic-repeat request 


ASCII: 


American National Standard Code for Information Interchange 


ASK: 


amplitude-shift keying 


ATM: 


asynchronous transfer mode 


AWGN: 


additive white Gaussian noise 


b/s: 


bits /second 


BER: 


bit error rate 


BISDN: 


broadband ISDN 


BPF: 


band-pass filter 


BSC: 


binary symmetric channel 


CAP: 


carrierless amplitude/phase modulation 


CCITT: 


Consultative Committee for International Telephone and Telegraph (N 
renamed the ITU) 


CDM: 


code-division multiplexing 


CDMA: 


code-division multiple access 


CELP: 


code excited linear predictive (model) 


CO: 


central office 


codec: 


coder/decoder 


CPFSK: 


continuous-phase frequency-shift keying 


CRC: 


cyclic redundancy check 


CW: 


continuous wave 


DAC: 


digital-to-analog converter 


dB: 


decibel 


dBW: 


decibel referenced to 1 watt 


dBmW: 


decibel reference to 1 milliwatt 


DC: 


direct current 


DEM: 


demodulator 


DES: 


data encryption standard 


DFT: 


discrete Fourier transform 


DM: 


delta modulation 


DMT: 


discrete multitone 


DPCM: 


differential pulse-code modulation 


DPSK: 


differential phase-shift keying 


DSB-SC: 


double sideband-suppressed carrier 


DS/BPSK: 


direct sequence/binary phase-shift keying 


DSL: 


digital subscriber line 


exp: 


exponential 


EDM: 


frequency-division multiplexing 


FDMA: 


frequency-division multiple access 


FEXT: 


far-end crosstalk 


FFT: 


fast Fourier transform 


FH: 


frequency hop 



FH/MFSK: 


frequency hop/M-ary frequency-shift keying 


FMFB: 


frequency modulator with feedback 


FSK: 


frequency-shift keying 


GMSK: 


Gaussian filtered MSK 


GSM: 


global system for mobile communication 


HDTV: 


high definition television 


Hz: 


Hertz 


BDFT: 


inverse discrete Fourier transform 


IF: 


intermediate frequency 


I/O: 


input/output 


IP: 


internet protocol 


IS-95: 


intermediate standard-95 


ISDN: 


integrated services digital network 


ISI: 


intersymbol interference 


ISO: 


International Organization for Standardizati 


ITU: 


International Telecommunications Union 


JPEG: 


joint photographic experts group 


LAN: 


local-area network 


LDM: 


linear delta modulation 


LMS: 


least-mean-square 


log: 


natural logarithm 


l°g2= 


logarithm to base 2 


logio! 


logarithm to base 10 


LPC: 


linear predictive coding (model) 


LPF: 


low-pass filter 


MAP: 


maximum a posteriori probability 


ML: 


maximum likelihood 


mmse: 


minimum mean-square error 


modem: 


modulator-demodulator 


MPEG: 


motion photographic experts group 


ms: 


millisecond 


/»s: 


microsecond 


MSK: 


minimum shift keying 


NCO: 


number-controlled oscillator 


NEXT: 


near-end crosstalk 


nrn: 


nanometer 


NRZ: 


nonreturn-to-zero 


NTSC: 


National Television Systems Committee 


OC: 


optical carrier 


OFDM: 


orthogonal frequency-division multiplexing 


OOK: 


on-off keying 


OSI: 


open systems interconnection 


PAM: 


pulse-amplitude modulation 
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PCM: 


pulse-code modulation 


PDM: 


pulse-duration modulation 


PG: 


processing gain 


PLL: 


phase-locked loop 


PN: 


pseudo-noise 


POTS: 


plain old telephone service 


PPM: 


pulse-position modulation 


PSK: 


phase-shift keying 


PSTN: 


public switched telephone network 


PWM: 


pulse-width modulation 


QAM: 


quadrature amplitude modulation 


i^oa: 


quality of service 


QPSK: 


quadriphase-shift keying 


RF: 


radio frequency 


rms: 


root-mean-square 


RS: 


Reed-Solomon 


DC 121 


Recommended standard-232 (port) 


RSA: 


Rivest- Shamir- Adelman 


RSC: 


recursive systematic conyolutional (code) 


RZ: 


return-to-zero 


s: 


second 


SDH: 


synchronous digital hierarchy 


SDMA: 


space-division multiple access 


SDR: 


signal-to-distortion ratio 


SNR: 


signal-to-noise ratio 


SUJNJiT: 


synchronous optical network 


STFT: 


short-time Fourier transform 


STM: 


synchronous transfer mode 


TC: 


time compression 


TCM: 


trellis-coded modulation 


TDM: 


time-division multiplexing 


TDMA: 


time-division multiple access 


TV: 


television 


UHF: 


ultra high frequency 


V: 


volt 


VCO: 


voltage-controlled oscillator 


VHF: 


very high frequency 


VLSI: 


very-large-scale integration 


W: 


watt 


WDM: 


wavelength division multiplexing 
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in geostationary orbit, 19 

historical notes, 26-29 

role of, 19 

"second generation", 515 
communication system 
common feature of, 3 
design of, 21 
elements of, 2 

noise analysis of, 3, 64, 523 
primary resources of, 3 



purpose of, 19, 88 
source of limitations, 248-252 
transition from analog to digital, 
183 

communication system designer, 23 
community-antenna television 

(CATV) system, 17 
commutator, 211 
compander, 203 
companding circuitry, 203 
companding law, fifteen-segment, 

426 

complementary error function, 255, 

256, 334 
complex envelope, 347, 727, 728, 

729, 730 
complex exponential Fourier series, 

717 

complex Fourier coefficient, 717 
complex least-mean-square (LMS) 
algorithm 
advantages of, 558 
limitations of, 558 
See also least-mean-square 
algorithm 
composite signal, 1 05, 149 
compound codes, 683-684 
compression algorithms, standard, 
8 

compression laws, 202-203 
compressor, 203 
computer communications, 2 
computer-generated data, 7 
conditional likelihood function, 

403-404 
conditional mean requirement, 200 
conditional probability, 706-707 
conditional probability density 

function, 320-321, 383, 

451-452, 710 
conditional probability of error, 

255, 256 
conditional probability of symbol 

error, 333 
confluent hypergeometric function, 

740-741 
confusion, 749 

conservation of time, 211-212 
constant angular velocity, 108 
constant envelope, 111 
constellation encoder, 444 
constrained optimization problem 

definition of, 437 

solving of, 609 



continuous AWGN channel, 

318-319 
continuous-phase frequency-shift 

keying (CPFSK), 381, 387 
continuous-phase frequency-shift 
keying (CPFSK) signal 
components of, 389 
deviation ratio of, 388 
phase of, 388 
representation of, 387 
continuous random variable, 594, 
708 

continuous source, 615 
continuous-time channel 

partitioning, 432—436 
continuous-wave (CW) modulation. 

20 

effects on reception, 130 

families of, 88-90 

principles of, 162 

techniques for, 130 
continuous- wave (CW) modulation 
system, 89-90 

comparison of, 132 

components of, 88-89 

noise in, 130 

performance of, 164 
control symbols 

in ASCII, 6 

for communication purposes, 6 
for printing of characters, 6 
conventional coherent binary FSK 
bit error rate expressions for, 
417 

with one-bit decoding, 417 
convolutional code, 654-656 
constraint length of, 655 
distance properties of, 663 
distinguishing feature of, 627 
maximum likelihood decoding 

of, 660-663 
performance of, 663 
use of, 654-656 
convolutional coding, 669 
convolutional encoder 
code tree for, 657-658 
input-output relation of, 660 
state of, 657-658 
trellis for, 658-659 
use of, 425 
convolution integral, 42—44, 71 8 
correlation coefficient, 342, 471 
correlation functions, 35-41 
correlation matrix, 40 
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correlation receiver, 326-328, 329 
correlative-level coding, 266-271 
basis of, 268 

generalized form of, 274-275 

idea ilJustrated, 267 

premise of, 267 
correlator, 24-25 

inputs of, 549-550 

outputs of, 319-322 
coset leaders, 638-639 
Costas loop 

generalization of, 454 

for phase recovery, 454 
Costas receiver 

consists of, 96-97 

phase control in, 97 

use of, 96-97 
cost function, 558 
covariance function, 35-36 
CPFSK. See continuous-phase 

frequency-shift keying 
Cramer-Rao bound 

denned as, 462 

modification of, 462 
CRC code. See cyclic redundancy 

check (CRC) code 
critical band, 234 
cross constellations, 369-370, 372 
cross-correlation functions, 40, 52 
cross-spectral densities, 52 
crosstalk, 21 

cause of, 279 

defined as, 501 

as impairment, 279 

types of, 279-280 
cryptanalysis, 742 

and authorized user, 742-743 

definition of, 746 

description of, 742 
cryptogram, 742-743 
cryptographic system 

classes of, 744 

classifications of, 759 

consists of, 743 

definition of, 743 

services of, 742 
cryptography, 617, 742 

and authentication problem, 743 

classifications of, 759 

data compression in, 749 

fundamental assumption, 745 

importance of, 759 

and secrecy problem, 743 
cryptology, 742 



crystal-controlled oscillator, 120 
cumulative distribution function, 
708 

cyclic code, 641-643 

advantage of, 641-642 

characteristics of, 652-654 

classes of, 652-654 

encoder for, 645-646 

generation of, 643 

properties of, 642 

in systematic form, 645, 646 
cyclic prefix, 441 
cyclic property, 642 
cyclic redundancy check (CRC) 
code 

for error detection, 652 
generator polynomials of, 653 

D 

damping factor, 160 
data-aided synchronization, 449 
data bits 

binary pattern of, 6 

for error detection, 7 
data communication, 7 
data compaction, 8, 575 

achieved by, 575 

assessing, 616 

schemes for, 575 
data compression, 7, 614-616 

in cryptography, 749 

forms of, 7-8 

idea of, 614 

as a lossy operation, 614-615 

reason for using, 615 

system components, 749 

techniques for, 218 
data compressor, 614 
data encryption, 617 
data encryption standard (DES), 

751-755, 759 
data encryption standard (DES) 

algorithm, 754 
data-modulated carrier, 500 
data multiplexers, 7 
data network, 11 
data signaling rate, 426 
data transmission system 

asynchronous versus 
synchronous, 7 

capabilities of, 446 

performance of, 293 
decision device, 259 
decision-directed mode, 290-291 



decision-directed recursive 

algorithm, 450 
decision errors, 210 
decision feedback, 270 
decision feedback equalization, 

291-293, 379, 430 
decision feedback equalizer (DFE) 

consists of, 292 

error propagation in, 292-293 

feedback section of, 292 

feedforward section of, 292 
decision-making criterion, 323 
decision-making device 

designs of, 277 

operation of, 25 
decision rule, 350-351, 357 

applying, 382 

denned as, 661 

as the MAP rule, 323-324 

as the maximum likelihood rule, 
324 

used by the coherent detector, 
497 

decision threshold, 194 
decision tree, 575-576 
decoder, 446 

condition for optimality, 
200-201 

consists of, 552 

function of, 552 

units of, 235 
decoding algorithms, 678 
decoding complexity, 684 
decoding decisions, 663 
decoding error, 660 
decoding process 

methods of, 693 

and pulse generation, 208 

requirements of, 261 
decoding rule, 660 
decoding spheres 

maximum number of, 600 

packing of, 600 
decoding window, 663 
decommutator, 211 
decorrelation time, 37 
decryption, 742 
de Forest, Lee, 27 
delay, average, 540 
delay power spectrum, 539 
delay spread, 539 

as a channel impairment, 553 

denned as, 540 

effect of, 556 
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delta function, 62 
property of, 716-717 
sifting property of, 190, 320 

delta modulation (DM), 218-221 
advantage of, 219, 237 
and digital pulse modulation, 
237 

principles of, 218-219 

quantization error of, 220 

simplicity of, 223 

and transmitters, 228 
delta-sigma modulation, 221-223 
demodulation, 20, 88 

method of, 99 

stages of, 491 
demodulation scheme, 132 
demodulator output, 92 
demultiplexer 

in receiver, 359, 445-446 

in transmitter, 444 
demultiplexing system, 125-126 
DES. See data encryption 
standarddetection 

and error correction, 628 

of a pulse signal, 248 
detector, 326, 349 
deviation ratio, 387 

definition of, 119 

versus modulation index, 119 
DFT. See discrete Fourier transform 
diagonal matrix, 443 
dibits, 276 

difference-frequency term, 158 
differential detector 

components of, 364 

tangent type, 364-365 
differential encoder 

of the binary wave, 414 

consists of, 421-422 

method used, 207 

requirement of, 207 
differential entropy, 593—597 
differential phase encoder, 415 
differential phase modulation, 422 
differential phase-shift keying 
(DPSK), 407, 414-417 

bit error rate of, 417 

generation and detection of, 415 

receiver, 416 

transmitter, 415—416 
differential pulse-code modulation 
(DPCM), 227-229 

basic idea of, 227 

and digital pulse modulation, 
237 



system comparison, 228 
and transmitters, 228 
differential quantization scheme, 

227, 228 
differentiator, 143 
Diffie-Hellman public key- 
distribution system, 756 
diffraction, 17-18 
diffusion, 749 

digital audio broadcasting, 448 
digital circuit technology, 189 
digital communication 

basic form of, 309-310 

and bit error rate, 24 

and design goals, 354 

elements of, 24-25 

receiver, 337 

reliability of, 23, 24-26 

requirements of, 23 

and system design, 22 

task of designer, 626 

use of, 21 
digital data transmission, 247 
digital filter, second-order, 

454-455 
digital hierarchy, 214 
digital modem 

bidirectional, 428 

capabilities of, 428 

and data rates, 429 

design constraints of, 426 

fundamental design philosophy 
of, 426 

one realization of, 426-427 

signaling scheme for, 427 

solution to design problems, 428 

theoretical basis for the design 
of, 429 
digital modulation schemes 

comparison of, 417-420 

probability of error, 417 

types of, 346 

using a single carrier, 417—420 

virtues of, 347 
digital modulation techniques 

operation of, 448 

types of, 345-346 
digital multiplexers, 214-215 

design problems, 215 

major groups of, 214 
digital multiplexing-demultiplexing 

operation, 214 
digital passband transmission 
system, 344 

assessing performance of, 335 



performance degradation of, 544 
digital PSTN, 426 
digital pulse modulation 

basic form of, 193 

feature of, 236 

transmission of, 183 
digital satellite communication, 419 
digital signals, 214 
digital signal zero (DS0), 214 
digital signature 

for electronic mail systems, 
758-759 

properties of, 759 

use of, 758-759 
digital subscriber line (DSL) 

as a growing application, 277 

line codes for, 280-281 

operational environment of, 277, 
447 

and twisted pairs, 277, 297 
versus voiceband modems, 446 

digital switch, 215, 446 

digital-to-analog converter (DAC), 
445 

digital transmission facility, 215 
digital wireless communication 

systems, 550-551 
Dirac delta function. See delta 

function 
direct broadcast satellite (DBS) 
simplicity and affordability of, 

517 

direct broadcast satellites (DBS) 

use of, 517 
direct frequency modulation, 

120-121, 396 
directive gain, 520 
directivity, 520 

direct matrix inversion (DMI), 558, 
559 

direct-sequence M-ary phase shift 

keying (DS/MPSK), 508 
direct-sequence spread binary 

phase-shift-keyed (DS/BPSK) 

signal, 490, 492 
direct-sequence spread spectrum 
with coherent BPSK, 490-193 
principles of, 480 
systems, 498 
Dirichlet's conditions, 715 
discrete cosine transform (DCT), 8 
discrete cosine transform 

coefficients, 8 
discrete Fourier transform (DFT), 

445 
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defined as, 442 

and digital signal processing, 443 
discrete memoryless channel, 
581-584, 629-631 
channel capacity of, 588 
defined as, 581-582 
discrete memoryless source, 570 
extension of, 572 
properties of, 568 
discrete multitone (DMT), 431, 
440-443, 444-446 
applications of, 446 
basic idea of, 441 
and multichannel modulation, 

447-448 
use of, 441 
discrete pulse-amplitude 

modulation (PAM), 259 
discrete pulse modulation, 259 
discrete random variable, 708 
discrete source, 615 
discrete-time, memoryless Gaussian 

channel, 597 
discrete-time channel, 291 
discrete-time convolution, 627 
discrete-time Fourier transform, 
185 

discriminator, 144 
discriminator output, 145, 155 
dispersive channel, doubly, 542 
distance transfer function, 666 
distortion, 2 

acceptable, 614 

methods of reduction, 103 

produced by, 102-103 

unavoidable, 611-612 
distortion, amplitude, 191 
distortionless baseband binary 

transmission, 261-262 
distortion measure, 199 
distribution function 

properties of, 708 

of a stationary random process, 
34 

diversity techniques, 544-547 
performance with, 546-547 
specialized techniques, 559 
"divide and conquer", 431 
DMI. See direct matrix inversion 
DMT. See discrete multitone 
Donald Duck voice effect, 99-100 
Doppler shift, 535 
Doppler spectrum, 540-541 
Doppler spread, 539, 541 
double-frequency term, 158 



double sideband-suppressed carrier 

(DSB-SC) modulated signal 

(wave), 95, 96 
double sideband-suppressed carrier 

(DSB-SC) modulation, 133, 

134 

definition of, 93 
generated by, 94 
transmission of sidebands, 163 
double sideband-suppressed carrier 
(DSB-SC) receiver 
compared to an AM receiver, 

136-137 
model of, 132-133 
doubly dispersive channel, 542 
down conversion, 105 
downconverter, 448 
downlink, 19, 514-515 
downstream data transmission, 

281-282 
DPSK. See differential phase-shift 
keying 

DS/BPSK waveform, 490, 492 
DSL environment, 447 
DS/MPSK system, 508 
dual code, 641 

duobinary code, modified, 281 
duobinary coding, 270 
duobinary conversion filter, 

268-269 
duobinary encoder, 267-268 
duobinary signaling scheme, 
267-271 
frequency response of, 268 
technique, 271-272 
duobinary technique, 274-275 
"dynamic" multipath environment, 
532-533 

E 

echo cancellation, 277-278 
comparison of schemes, 278-279 
mode of operation for, 277-278 

echo canceller 

in transceiver, 278-279 
use of, 516 

effective aperture, 521 

effective radiated power referenced 
to an isotropic source (EIRP), 
521 

Einstein-Wiener- Khintchine 
relations, 46 

EIRP. See effective radiated power 
referenced to an isotropic 
sourceelastic store, 215 



electromagnetic interference (EMI), 
17 

electron beam, 4 
elementary event, 704 
encoded text, 6 
encoder 

condition for optimality of, 
199-200 

functional units of, 235 

main parts of, 551 

operation of, 646 

states of, 659, 667 
encoding process 

operations of, 9 

steps of, 551 

use of, 203-204 
encryption, 742 
enemy cryptanalyst 

forms of attack by, 745 

intrusion of, 743, 744 
energy gap, 99 
energy signals, 312 
energy spectral density, 48, 353 
ENIAC, 28 
ensemble average 

autocorrelation function, 284 

estimation of, 41 

parameter, 75 

substituting time averages for, 41 
entropic coding redundancy, 9 
entropy 

conditional, 584 

definition of, 569 

formula for, 568 

properties of, 570-571 
entropy, conditional, 584 
envelope 

denned as, 730 

and phase components, 67-69 

types of, 730 
envelope delay, 16 
envelope detection, 102-103, 131 
envelope detector, 123, 143 

consists of, 92 

found in, 92 

loss of message in, 138 

need for, 406 

performance of, 137 

signal comparison, 141-142 
envelope distortion, 90-91 
equalizer, 191 
equiprobable symbols, 571 
equivalent noise temperature, 61, 

524-525 
ergodic process, 41—42, 51 
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error 

minimization of, 551 

possible kinds, 254 

probability of, 497-^199 
error burst, 652 
error control, 626 
error-control code 

theory of, 485 

types of, 627 
error-control coding 

classes of, 626 

for reliable communication, 567 

techniques, 626 

techniques for, 683, 693 

use of, 626, 627 
error-detection bit, 7 
error-free communication, 568 
error function, complementary, 255 
error minimization, 551 
error pattern, 635 

error probabilities, conditional, 352 
error propagation 

elimination of possibility, 273 

phenomenon, 270 

property, 745 
error rate, 253 
error signal 

calculation of, 457 

definition of, 288, 453 

for timing recovery, 457 

use of, 557 
error-syndrome vector, 635-636 
error threshold, 209-210 
error vector, 635 
estimation procedure, 517 
Euler's formula, 453 
excess bandwidth factor, 441 
excess mean-square error, 291 
excitation generator, 551, 552 
excitation time, 718 
expander, 203 
expansion laws, 203 
exponential law, 193 
extended code 

average code-word length, 578 

use of, 577 
extended prefix code, 578 
extended source, 572 
extended-threshold demodulators, 

152, 153 
extraction, 261 
extrinsic information, 679 
eye opening, 293 
eye pattern 



definition of, 293 
as an experimental tool, 293 
interpretation of, 293 
and performance information, 
293 

F 

facsimile (fax) machine 
basic principle of, 6 
purpose of, 6 

in a receiving mode of operation, 
420 
fade rate, 541 
fading channel 

characteristics of, 541 
effects of, 545 
fading multipath channel, 536-539 
far-end crosstalk (FEXT), 279-280 
Famsworth, Philo T., 27 
fast Fourier transform (FFT) 

algorithm, 443-444 
fast-frequency hopping, 502-503, 
504 

FDM. See frequency division 
multiplexing 

FDMA. See frequency division 
multiple access 

FDMA system, 516 

FDM system 
block diagram of, 105-106 
modulation steps in, 107 

FEC. See feed-forward error 
correction 

feedback shift register, 480, 481 

feedback system, second-order, 160 

feed-forward error correction 
(FEC), 628, 629 

Fessenden, Reginald, 27 

FH/MFSK system 
fast versus slow, 502-503 
jamming effect on receiver, 502 
symbol error in, 502 

field-power pattern, 521 

figure of merit, 134 
for amplitude modulation, 136 
definition of, 132, 193 
for frequency modulation, 147 

filtering, 128, 208-209 

filtering scheme, 100 

fine synchronization, 493 

finite-duration impulse response 
(FIR) filter, 379 

finite-state machine, 654 

fixed channel input, 582 



fixed channel output, 582 
fixed modulation scheme, 627 
fixed point-to-point links, 18-19 
fixed scatterers, 536 
flat-fading channel, 71-72 
flat-flat channel, 542 
flat Rayleigh fading channel, 554 
flat-top samples, 191 
Fleming, John Ambrose, 27 
flip-flops, 646 

flyback. See horizontal retrace 

FM demodulator 
with negative feedback, 153 
and oscillator types, 152 

FMFB demodulator, 152, 153, 154 

FMFB receiver, 154 

FM receiver 

breaking point of, 149 
interference suppression in, 

148-149 
model of, 142-143 
noise analysis of, 146 
noise in, 142 
threshold effects in, 152 

FM signal 
average power of, 1 15 
complex envelope of, 1 14 
demodulation of, 121-124 
desirable properties, 397 
detection of, 397 
distinguishing from AM signal, 
109 

effective bandwidth for, 117-119 
fundamental characteristic of, 
110 

generation of, 120-121 

side frequencies of, 117 

spectral analysis of, 110 

spectrum of, 115 

in theory and practice, 117 
FM signal, single-tone, 112-113 
FM stereo 

multiplexing, 124-126 

specification of standards, 124 

transmission, 124 
FM system 

emphasis in, 154-156 

nonlinear effects in, 126-128 

See also frequency modulation 
(FM) system 
FM threshold effect, 149-152 
FM threshold reduction, 152-154 
FM wave 

bandwidth requirement, 118 
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with reduced modulation index, 
152-153 

forward error-control coding, 628, 
668 

forward error correction {FEC}, 
626 

forward estimation, 681 
forward link, 547 
Fourier analysis, 715-720 
Fourier series expansion, 317 
Fourier series representation, 114 
Fourier transform 

definition of, 715 

inverse, 715 

of periodic signals, 717-718 

properties of, 716 

theory of, 716 
fractionally spaced equalizer (FSE), 

287 
frame, 552 

make-up of, 5 

method of synchronization, 
215-216 

structure, 547 
frame packing, 9 
frame-packing unit, 236 
free distance, 663 
free propagation 

channels based on, 15 

types of, 15 
free-space loss, 522 
free-space propagation model, 

518-523 
frequency demodulation 

defined as, 121 

methods of, 121 
frequency deviation, 110, 152 
frequency-discrimination method 

stages of, 98-99 

use of, 100 
frequency cuscrirninator, 121, 
124 

consists of, 121-122 
input, 149 
requirements of, 99 
frequency diversity, 544-545 
frequency division duplexing 

(FDD), 547 
frequency division multiple access 

(FDMA), 513, 516 
frequency-division multiplexing 
(FDM) 
defined as, 20-21, 105 
method of modulation in, 106 



frequency-domain description, 444, 
720 

frequency down converter, 105, 
516 

frequency flat, 541 
frequency-flat channel, 542 
frequency-hop M-ary frequency 

shift-keying (FH/MFSK), 508 
frequency hopping, 500 
frequency-hop spread spectrum, 
499, 500 

communication systems, 500 

principles of, 480 
frequency-modulated wave, 413 
frequency modulation (FM), 20, 
27, 729 

capability of, 165 

cases of, 111 

characteristic of, 149 

definition of, 108-109 

direct, 120-121 

and mixing, 500 

as a nonlinear process, 109 

theory of, 126 
frequency modulation (FM) system 

noise analysis of, 142-147 

similarities to PPM system, 193 
frequency multiplication ratio, 
121 

frequency multiplier 

consists of, 120 

diagram of, 120-121 
frequency parameters, 128 
frequency response 

choice of, 155 

to denote, 44-45 
frequency reuse, 530 
frequency-shift keying (FSK) 

basic signaling scheme, 344-345 

and design of modems, 421 

and frequency modulation, 345 

represented by, 464 
frequency-shift keying (FSK) 

schemes, 418 
frequency-shift keying (FSK) signal, 
386 

frequency translation, 103, 

103-105 
frequency up converter, 105 
Friis formula, 526 
Friis free-space equation 

defined as, 522 

used for, 522 
FSK. See frequency-shift keying 



full amplitude modulation, 

162-163 
full-cosine rolloff characteristic, 

266 

full-duplex link, 628 
functional 

defined as, 54 

versus function, 54 
fundamental frequency, 717 
fundamental inequality, 571 

G 

gain, 720 
gap, 432 

Gaussian assumption, 498 
Gaussian channel, 597 
Gaussian-distributed random 

variable, 54 
Gaussian distribution, 54, 72-73 
Gaussian filter, 397 
Gaussian-filtered minimum shift 
keying (GMSK) 
as a special kind of binary 

frequency modulation, 398 
undesirable feature of, 398 
Gaussian-filtered minimum shift 
keying (GMSK) modulator 
frequency shaping pulse of, 397 
and intersymbol interference, 
398 

Gaussian-filtered minimum shift 
keying (GMSK) signal, 397 

power spectrum of, 400 

spectral compactness of, 
398-400 
Gaussian-filtered MSK, 396-400 
Gaussian function, 397 
Gaussianity, 75 
Gaussian model, 55 
Gaussian process, 54-58 

definition of, 57 

mathematical justification, 55-56 
in the study of communications, 
55 

useful properties of, 56-58 

virtues of, 55 
Gaussian random variable, 54, 58 
generator equation, 634 
generator polynomial, 645 

of a cyclic code, 643 

definition of, 656 
geometric mean, 436 
geometric representation of signals, 
311 
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geometric signal-to-noise ratio, 436 
geostationary satellite 

communications system, 

514-515 
global coverage, 512 
Global System for Mobile 

Communications (GSM), 548 
frame efficiency of, 548 
wireless communication system, 

548 
glottis, 4 

GMSK. See Gaussian-filtered 

minimum shift keying 
Gold's theorem, 505 
Gold sequences (codes) 
class of, 505 

correlation properties of, 507 
good codes, 631 

Gram-Schmidt orthogonalization 

procedure, 315-317 
granular noise, 220 

and distortion, 221 

versus quantization noise, 221 
Gray coding scheme, 422—423 
Gray encoder, 276 
Gray-encoded dibits, 363 
GSM. See Global System for 
Mobile Communications 
guard bands, 513 
guard interval, 441 
guard time, 547 
guided propagation 

channels based on, 15 

types of, 15 

H 

half-cycle cosine pulse, 389 
half-cycle sine pulse, 389-390 
half-duplex link, 628 
Hamming distance, 637, 661, 
666 

Hamming single-error correcting 

code, 653 
Hamming weight, 637, 666 
handover, 530 
hard-decision coding, 630 
hard-decision decoders, 629-630 
hard-decision demodulation, 669 
hard decisions, 630 
harmonic distortion, 112 
harmonic structure, 4 . 
head end, 17 
hearing mechanism, 4 
Hermitian transposition, 443 
Hertz, Heinrich, 26-27 



heterodyning function, 128 
hexagonal cellular geometry, 531 
high-performing CAP system, 375 
Hilbert transform, 374, 408, 
723-725 

properties of, 725 

of a signal, 724 
Hilbert-transform pair, 376, 724 
Hockham, G. A., 29 
hop rate, 501 
horizontal retrace, 5 
host, 13 
Huffman code 

algorithm used to synthesize, 
578 

as a class of prefix codes, 578 

drawback of, 580 

nonuniqueness of, 579 
Huffman coding, 578-580, 616 

basic idea of, 578 

compared to Lempel-Ziv 
algorithm, 581 

and data compression, 8 

as entropic coding, 8 
Huffman decoding, 8-9 
Huffman encoding process, 578, 
579 

human auditory system, 234 
human communication, 4 
hybrid-modulated signal, 123 
hybrid modulation process, 374 
hybrid transformer 

definition of, 278 

simplified circuit of, 278-279 

I 

/-channel, 97 

ideal baseband pulse transmission, 
262 

ideal delay element, 268 
ideal envelope detector, 135 
ideal frequency cUscrirninator, 124 
ideal narrowband filter, 45 
ideal Nyquist channel, 262-264, 
265-266 

difficulties of, 263-264 

use of, 263-264 
ideal sampled signal, 184 
ideal slope circuit ; 

characterized by, 121-122 

frequency response of, 122 
ideal system, 601 
identity matrix, 329 
image interference, 129 
impossible event, 704 



impulse function, 62 
impulse noise, 446 
impulse response, 656, 718 
index of performance, 224 
indirect frequency modulation, 

120-121 
individual demodulators, 106 
infinite bandwidth, 602 
information, 2 

information-bearing signal, 31-32, 
88 

in the digital domain, 277 
multiplying by the PN signal, 
488 

information capacity, 598, 616 
of a channel, 597-598 
defined as, 23, 598 
evaluation of, 598 
increasing of, 599 

information capacity theorem, 616 
application of, 607 
argument for, 599-600 
as a colored noise channel, 607 
and Gaussian channels, 597 
implications of, 601-603 
system parameters of, 599 
water-filling interpretation of, 
610 

information-theoretic concepts, 
572 

information theory 
fundamental limits in, 567 
important result of, 591 
as a mathematical discipline, 
567 

Shannon's landmark paper, 567 
information transmission, 581 
information vector, 633 
inner conductor, 17 
inner product, 313, 314 
innovation symbol, 581 
in-phase channel, 408 
in-phase coherent detector, 97 
in-phase component, 93 

power spectral density of, 
395-396 

properties of, 65-66 

representation of, 67—69 
in-phase noise component, 131 
input alphabet, 582 
input signal-to-noise ratio, 134 

definition of, 131 

equation for, 497 
insertion loss, 16 
instantaneous codes, 577 
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instantaneous frequency, 110 

definition of, 163 

equation for, 108 
instantaneous sampling, 184 
integration 

beneficial effects of, 221-222 

as a linear operation, 223 
interface, 11 
interference 

average power of, 495 

effect of, 490 

and fading, 71-72 

strength of, 148-149 

as unintentional or intentional, 
479 

interference suppression, 148-149 
interframe redundancy, 9 
interlaced fields, 5 
interlaced raster scan, 5 
interleaver 

definition of, 674 

types of, 674 

use of, 675 
intermediate frequency (IF), 128 
intermediate frequency (IF) band, 
18 

Internet, 13-14 

architecture of, 13-14 

evolution of, 28-29 

growth of, 29 

protocols for, 13-14 
Internet architecture 

functional blocks of, 13-14 
Internet protocol (IP), 13-14 
Internet Service Provider (ISP), 420 

and communication between 
PSTN, 425 

and public switched telephone 
network (PSTN), 420-425 

and voice modems, 420-422 
interpixel redundancy, 9 
interpolation formula, 186 
interpolation function, 186, 427 
intersymbol interference (ISI), 
259-261, 398 

and bit errors, 247 

as channel impairment, 379 

condition of, 282-283 

under designer's control, 268 

as a dominant impairment, 279 

effects of, 294, 296 

as a form of interference, 296 

minimizing effects of, 260 

and noise presence, 294 

overcoming effects of, 441 



in peak distortion, 288 

and timing error, 266 

as an undesirable effect, 267 

intrinsic information, 679 

invariance, 331 

inverse discrete Fourier transform 

(IDFT), 442, 445 
inverse Fourier transform, 186, 

715 

inverse mapping, 589 
inverse-square law, 519 
irreducible polynomial, 505 
irregular codes, 691 
irregular interleaves, 691-692 
irregular LDPC code, 692 
irregular turbo code, 691, 692 

J 

jammer, 493 

strategy of, 495 

types of, 508 

waveforms of, 508 
jammer, barrage noise, 508 
jammer, multitone, 508 
jammer, pulse noise, 508 
jammer, single-tone, 508 
jamming margin, 499 
jamming signal, 488 
jamming waveforms, 488 
jitter, 208 

joint distribution function, 33, 709 

joint moments, 713-714 

Joint Photographic Experts Group 

(JPEG), 8 
joint probability, 706, 707 
joint probability density function, 

594, 709-710 
joint probability distribution, 583 
JPEG image coding standard, 8 

K 

Kao, K. C, 29 
keys, 756 

key-schedule calculation, 753, 754 
keystream, 744, 745 
Kotel'nikov, V. A,, 27 
Kraft-McMillan inequality, 

576-577 
Kummer's differential equation, 

740 

L 

Lagrange multipliers 
method of, 437 
use of, 609 



laser, 29 
layer, 11 

layered architecture, 11 
layer-to-layer interface, 13 
least-mean-square (LMS) 

algorithm, 288-290, 557 
for adaptive equalization, 

288-289 
and combined use, 297 
equations for, 289 
for linear adaptive prediction, 
226 

popularity of, 226, 227 
similarities, 290 
simplification of, 289 
summary of, 289 
uses of, 292 

using matrix notation, 289 
Leibniz's rule, 257 
Lempel-Ziv algorithm, 8, 616 

compared to Huffman coding, 
581 

definition of, 580 
encoding process performed by, 
580 

standard for file compression, 
581 

Lempel-Ziv coding, 580-581 
light, 6 

likelihood functions, 322 

linear adaptive prediction, 225-227 

linear array signal processor 

to design, 554 

for the receiver, 554 

requirements of, 554 
linear block code 

basic property of, 634 

classes of, 653-654 

decoding procedure for, 639 

definition of, 632 

mathematical structure of, 
632-633 

minimum distance of, 637 

standard array of, 638 
linear combiner, 547 
linear delta modulator, 221, 

232-233 
linear diversity combining 

structure, 545 
linear equalization, 379, 556 
linear function, 54 
linearity property, 642 
linear modulation 

definition of, 93 

examples of, 163 
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linear modulation (Continued) 

forms of, 93-94 

types of, 93 
linear modulation systems, 729 
linear prediction, 223-227 
linear predictive coding (LPC), 551 
linear pre-emphasis and de- 
emphasis filters 

applications in, 157 

use of, 157 
linear receiver, 248 

design of, 283 

performance of, 132 

using coherent detection, 132 
linear system, 718 
linear time-invariant filter 

defined as, 250-251 

impulse response of, 44 

as a matched filter, 250-251 

use of, 248 
linear time-invariant system, 

719-720 
line codes 

candidates for, 281 

comparison of, 281 

for electrical representation, 
204-207 

power spectra of, 206 

selection of, 280-281 

types of, 205-207 

use of, 204-207 
line-scanning frequency, 5-6 
link budget, 518 
link budget analysis, 517 
link budget balance sheet, 517 
Lloyd-Max quantizer, 198, 

200-201 
loading problem, 438 
loading process, 438 
local loop, 420 
local oscillator, 97 
Lodge, Oliver, 27 
logarithmic function, 322 
log-likelihood function, 452 
defined as, 322 

denned for AWGN channel, 325 

relationship of, 322 
loop filter, 157, 159-160 
loop-gain parameter, 159 
lossless compression 

definition of, 7-8 

for digital text, 8 

versus lossy compression, 8 
lossless data compression, 575 



lossy compression 
definition of, 8 
the preferred approach, 8 

low-density parity-check (LDPC) 
codes, 683-686 
advantages of, 684 
block length of, 689 
construction of, 684-685 
decoding algorithm, 690 
decoding of, 689-690 
initialization of, 690 
minimum distance of, 689 
shared properties of, 693 
statistical analysis of, 689 
steps of, 690-691 
use of, 684 

low-noise amplifier, 515 

low-weight code words, 684 

LPC. See linear predictive coding 

Lucky, Robert, 28 

luminance signal, 6 

M 

magnitude response, 45, 608, 719 
magnitude spectrum, 715 
main lobe, 368, 720 
Manchester code, 207, 281 
many-to-one mapping, 8 
MAP decoder, 678 
Marconi, Guglielmo, 27 
marginal densities, 710 
marginal probability distribution, 
583 

Markov process, 678 
M-ary digital modulation 

techniques, 419-420 
M-ary frequency-shift keying 

(MFSK), 398^00 
consists of, 401 

for frequency hopping systems, 
500 

property of, 400 

M-ary FSK signal 
bandwidth efficiency of, 

401^02 
bandwidth requirements, 401 
orthogonal signals of, 401—402 
power spectra of, 401 
spectral analysis of, 401, 402 

M-ary PAM system 
in a channel bandwidth, 276 
consideration of, 276 
design complexity of, 277 
power requirements of, 276-277 



M-ary PSK 

comparison of, 419 
likelihood function for, 452 
power-bandwidth requirements 

for, 419-420 
signal constellations of, 365, 
420 

similar spectral and bandwidth 
characteristics, 419 

special case, 365 

symbol duration of, 367 

symbol error equation for, 
365-366 
M-ary PSK signal 

bandwidth efficiency of, 368 

baseband power spectral density 
of, 367 

power spectra of, 367 

as spectrally efficient, 402 
M-ary PSK systems, 449-450 
M-ary QAM 

detection for, 371 

functions in, 369 

performance of, 420 

symbol error probability for, 371 

transmitted energy in, 371 
M-ary QAM signal, 372 
M-ary quadrature amplitude 

modulation (QAM), 369-373 
M-ary signal, 402 
M-ary signaling scheme, 345-346 
M-ary system, 276 
masking threshold, 9, 234, 

234-235 
matched filter, 248-252, 286 

in the frequency domain, 251 

output, 406 

properties of, 251-252 
matched filter receiver 

correlation and, 326-327 

detector part of, 328 
mathematical models 

classes of, 31 

in probabilistic terms, 31 
matrixer 

difference signal generation, 125 

sum signal generation, 125 
maximal-length sequence 

autocorrelation function of, 482 

balance property of, 482 

choosing a, 484 

defined as, 482 

properties of, 482-484 
maximal-ratio combiner, 547 
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maximal ratio combining principle, 
550 

maximum a posteriori probability 

(MAP) detection, 678 
maximum a posteriori probability 

(MAP) rule, 323-324 
maximum likelihood decision rule 
for an AWGN channel, 325 
purpose of, 325 
maximum likelihood decoder, 
322-326 
for computation, 324 
defined as, 661 

as an implementation device, 324 
theory of, 660-661 
maximum likelihood decoding rule, 
661 

maximum likelihood detection, 

330, 337 
maximum likelihood detectors, 435 
maximum likelihood estimation 

of the carrier phase, 453-458 

for problem solving, 449 
maximum likelihood rule, 324 
maximum likelihood signal 

detection, 346 
maximum-power transfer theorem 

applying, 61 

use of, 61 
Maxwell, James Clerk, 26 
mean, 35 

mean Doppler shift, 541 
mean functions, 35-39 
mean output noise power 

definition of, 139 

equation for, 141 
mean output signal, 139 
mean-square distortion, 199 
mean-square error, 285 

as the cost function, 558 

definition of, 284 
mean-square error criterion 

for receiver design, 283 

uses of, 288 
median signal strength, 530 
melodic structure, 4 
memory less channel, 321 
memoryless Gaussian channel, 

discrete-time, 597 
memoryless quantizer, 194 
Mersenne prime length sequences, 

485 
message, 155 
message bandwidth, 90 



message point, 322-323 
message polynomial, 643 
message signal, 132-133 

description of, 2 

generation of, 2 
message source, 348 
message spectrum 

for negative frequencies, 103 

requirement of, 99 
message vector, 660 
method of steepest descent, 

225-226 
microcells, 531 
microphone, 15-16 
Middleton, D., 27 
minimum average energy, 332 
minimum distance 

considerations of, 637-638 

definition of, 637 
minimum distance decoder, 661 
minimum energy signals, 331-332 
minimum energy translate, 332 
minimum mean square error 
(MMSE), 558 

criterion for, 558 

equilizer, 285 

receiver, 286-287 
minimum shift keying (MSK), 387 

as a form of binary FSK, 361 

signal-space diagram of, 
389-392 

minimum shift keying (MSK) signal 

power spectra of, 360 
mixer 

consists of, 103-104 

function of, 129 

operation of, 105 
mobile radio, 18, 529 
mobile radio channel 

capability of, 18 

as a linear time-varying channel, 
18 

propagation effects of, 18 
mobile switching center, 530 
mobility, 18 
modem, 7 

configuration of, 421 

as a conversion device, 420 

design of, 421 

and the Internet, 420 

portions of, 420 
modem, facsimile, 420 
modem configuration, 421 
modified Bessel equation, 738 



modified Bessel function, 737-739 
modified duobinary code, 281 
modified duobinary coder 

responses of, 272, 273 

useful feature of, 272-273 
modified duobinary conversion 

filter, 273 
modulated signal (wave), 95 
modulating signal (wave), 88 
modulation, 19 

operations of, 628 

stages of, 490 
modulation format, 101-102 
modulation index 

defined as, 110 

restriction of, 112 

small values of, 119 

values of, 117-118 
modulation process, 19-21 

classification of, 20 

definition of, 88 
modulation scheme, bandwidth- 
conserving, 354 
modulation system, binary-coded, 
197 

modulator-demodulator, 420 

modulo-2i7 correction logic, 364 

moments, 712 

Morse, Samuel, 26 

Morse code, 26, 574 

Motion Picture Experts Group 

(MPEG), 9, 234 
moving -coil receiver, 15-16 
MPEG-1 audio coding standard, 9 

capabilities of, 234 

operation of, 235 

performance of, 234 

suitable for, 10 
MPEG-1 video coding standard, 9 
MPEG audio coding standard, 234, 
237 

MSK receiver, 394, 395 
MSK signal, 391 

characteristics of, 396 

demodulation of, 394 

detection of, 394 

error probability of, 392 

generation of, 394, 396 

possible forms of, 390 

power spectra of, 394-396, 398 

properties of, 396 
MSK system, 391 
MSK transmitter, 394, 395 
(i-law, 202-203 
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multibeam antennas, 514 
multichannel data transmission 

system, 433, 434 
multichannel modulation, 440-441 

basic idea of, 431 

form of, 431, 465 
multichannel transmission system, 

437-438 
multilevel encoding, 348, 378 
multiloop feedback circuit, 480 
multipath 

physical phenomenon of, 513 

presence of, 513 
multipath autocorrelation profile, 
537 

multipath channel 

as a channel impairment, 553 

classification of, 541-542 

frequency selection, 541-542 

model of, 71-72 

statistical characterization of, 
535-542, 554 

type of fading exhibited, 72 
multipath component, 549 
multipath intensity profile, 539 
multipath phenomenon 

forms of, 532-533 

in a mobile radio environment, 
18 

nature of, 532 
multiple access 

basic types of, 513-514 

versus multiplexing, 513 
multiple-access interference (MAI), 
548 

multiple-access system 

goal of, 549 

interference, 548 
multiple-access techniques 

common feature of, 514 

defined as, 513-514 

ideas behind, 513, 514 

and shared communication 
resources, 513 
multiple-receiver combining 

techniques, 544 
multiplexed signal, 125-126 
multiplexed systems, 347 
multiplexer, 446 
multiplexing, 105 

definition of, 20 

of digital signals, 214 

types of, 20-21 
multiplier, 157, 646 



multi-pulse excited LPC, 551-552 
multitone jammer, 508 
multiuser communications, 512 

environment, 396 

types of, 559 
music 

as a source of information, 4 

structures of, 4 
musical signals 

and channel bandwidth, 4 

versus speech signals, 4 
mutual information, 584-585 

for continuous ensemble, 
593-597 

defined as, 596 

properties of, 585-587, 596 

in the Shannon model, 746 

N 

narrowband AM signal (wave), 

112-113 
narrowband FM signal (wave), 

111-112, 113 
narrowband FM waves, 112-113 
narrowband frequency modulation, 

111-113 
narrowband noise, 64 
characterization of, 64 
components of, 64, 65-66 
effects of, 64 

representation and coordinate 

system for, 67-68 
representation of, 64-66, 67-69 
narrowband noise analyzer, 64-66 
narrowband noise synthesizer, 
64-66 

narrowband phase modulator, 120 
narrowband process, 65-66 
National Television System 
Committee (NTSC), 6 
natural frequency, 160 
N-dimensional Euclidean space 

angles in, 312 

lengths of vectors, 312 

vectors in, 311, 312 
N-dimensional vector, 311 
near-end crosstalk (NEXT), 

279-280, 380 
nearest neighbor condition, 200 
near-far problem, 548, 549 
negative-going click, 150 
network, 10 

network, interconnected, 13 
network resources, 11, 14 



90 degrees rotational invariance, 

422-423 
noise 

absence of, 261 

calculations, 61 

in communications systems, 58 

definition of, 3 

effect of, 3 

minimizing the effects of, 157, 
260 

presence of, 589 
signals in, 322-326 
sources of, 3, 58 
as unwanted signals, 58 
noise analysis 

comparison of, 163-164 
at the receiver, 523 
noise calculations, 61 
noise channel, colored, 607-611 
noise enhancement, 283 
noise equivalent bandwidth, 
722-723 
for bandpass filters, 723 
defined as, 723 
noise figure, 523-526 
noise-free estimates, 545 
noise jammer, barrage, 508 
noise jammer, partial-band, 508 
noise margin, 281 
noise masking, 234 
noise power, 3, 61 
noise power, average 

outputcalculation of, 151 
determining, 145-147 
noise-quieting effect, 147 
noise-to-mask ratio (NMR), 234 
noise vector 

covariance matrix of, 330 
particular realization of, 323 
statistical characteristics of, 330 
noisy receiver model, 130 
noisy resistor, 60 
noncoherent binary DPSK, 407 
noncoherent binary frequency-shift 
keying (FSK), 407, 413-414 
bit error rate for, 414 
consideration of, 413 
expressions for, 417 
noncoherent M-ary FSK detector, 
500 

noncoherent matched filter, 406 
noncoherent orthogonal 
modulation, 407-409 
noise performance of, 407 
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receiver for, 408 
special case of, 414 
noncoherent receiver, 405-406, 
409 

nondata-aided early-late delay 

(NDA-ELD) synchronizer, 458 

nondata-aided recursive algorithm, 
455 

nondata-aided synchronization, 
449 

nonflat channel, 542 
nonlinearity 

basic forms of, 126 

effects of, 126 

the presence of, 126 
nonlinear modulation process, 163 
nonlinear pre-emphasis and 

de-emphasis techniques, 157 
nonredundant coding, 421 
nonreturn-to-zero level encoder, 
359 

nonsystematic code, 656 
nonuniform quantizer, 202-203 
normalized Gaussian distribution, 

54-55, 56 
normalized transmission 

bandwidth, 164 
North, D. O., 27 
North American digital TDM 

hierarchy, 214-215 
Norton equivalent circuit, 60 
Noyce, Robert, 28 
NRZ binary data, 398 
NTSC system, 6 
null event, 704 

null-to-null bandwidth, 368, 721 
number-controlled oscillator 

(NCO), 458 
Nyquist, Harry, 27 
Nyquist bandwidth, 262 
Nyquist criterion, 262 
Nyquist interval, 186-187 
Nyquist rate, 186-187, 262 

O 

observable element, 321 
observation space, 324, 325-326 
observation vector, 321, 331 
octaphase-shift-keying, 365-366 
octet, 7 

offset QPSK, 362 
onboard switching, 516 
one-time pad, 747 
on-off level encoder, 384 



on-off signaling, 205 

open systems interconnection (OSI) 

reference model, 11 
optical communication, 29 
optical fiber 

advantages of, 17 

consists of, 17 

properties of, 17 

as a transmission medium, 17 
optical transmission system, 15 
optimization problem, 438-440 
optimum CAP receiver, 379 
optimum decision rule, 323-324 
optimum equalizer, 288 
optimum filter 

consists of, 378-379 

impulse response of, 250-251 
optimum information capacity, 610 
optimum in-phase filter, 378-379 
optimum linear receiver, 282-287 

interpretation of, 286 

and transmitter, 286 
optimum quadratic receiver, 

403-405 
optimum quadrature filter, 

378-379 
optimum quantization problem, 

199-201 
optimum receiver 

as a correlation receiver, 
326-327 

design of, 310 

detector part of, 328 

subsystems of, 326-327 
optimum receiver subsystems, 

326-327 
optimum threshold, 257 
orthogonal frequency-division 
multiplexing (OFDM), 
447-448 

applications of, 448 

techniques for broadcasting, 344 

use of, 447-448 
orthonormal basis functions, 311, 
315, 494 

as a desirable property, 440-441 

for MSK, 390 

shortcomings of, 441 
orthonormal matrix, 443 
oscillator, crystal-controlled, 120 
OSI model, 11-13 
outer conductor, 17 
output alphabet, 582 
output current, 4-5 



output noise power, 523 
output signal-to-noise ratio 

of AM receiver, 136 

calculation of, 151 

versus carrier-to-noise ratio, 151 

definition of, 131, 145 

determining, 133 

equation for, 139, 497 

to evaluate, 135 

improvement factor in, 155-156 
increasing of, 155 
of a uniform quantizer, 197 
overmodulated, 90 

P 

packet switching 

network, 11 

principle of, 1 1 
pairwise error probability, 334 
PAM. See pulse amplitude 

modulation (PAM) 
PAM signal 

generation of, 188-189 

performance of, 191 

sampling of, 189 

transmission of, 191 

waveform of, 188-189 
PAM system, 191 
parallel encoding scheme, 676 
parallel-to-serial converter, 445 
parity bit, 7, 632 
parity-check equations, 634 
parity-check matrix, 634 
parity-check polynomial 

defined as, 644 

reciprocal of, 645 
partial-band noise jammer, 508 
partial-response signaling, 269, 

274-275 
partial-response signaling scheme, 
267 

achieved by, 274-275 

classes of, 275 

useful characteristics of, 275 
partitioning, 670 
passband basis functions 

properties of, 434-436 

time variations of, 373 
passband data transmission, 344 

alternative techniques for, 465 

applications of, 344 

communication channel used for, 
344 

over nonlinear channels, 345 
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passband data transmission systems 
determining the bandwidth 

efficiency of, 348 
goal of, 346 

passband in-phase filter, 375, 378 

passband in-phase pulse, 375 

passband line code, 344 

passband modulation, 729 

passband pulse, 375 

passband quadrature pulse, 374 

passband signaling waveform, 
729 

passband transmission model, 

348-349 
path loss, 522 

pattern matching operation, 615 
PCM. See pulse-code modulation 
peak distortion, 288 
peak pulse signal-to-noise ratio 

defined as, 248-249, 250 

of a matched filter, 251 
peer process, 13 
percentage modulation, 90 
perception, 4 
perceptual coding, 9 
perfect security, 746-747 
periodicity, 71 8 
periodic signals, 717-718 
periodogram, 51 
persistence of vision, 5 
personal computers (PCs), 6, 6-7 
"phase and gain adjusters", 550 
phase continuity, 388 
phase correction, 365 
phase decisions, 394 
phase demodulation, 492 
phase-difference computer, 364 
phase discriminator, 97 
phase distortion 

and the human ear, 99-100 

presence of, 99-100 
phase error 

defined as, 158 

effect of, 99 
phase-error generator, 459 
phase-locked loop, 121 

complexity of, 159-160 

components of, 157-160 

limitation of, 159-160 

loop filter in, 158 

model of, 158-160 

simplest form of, 159-160 

understanding, 157, 158 

use of, 157-160 



phase-locked loop demodulator, 
152 

and threshold extension capacity, 
154 

as a tracking filter, 154 
phase-locked loop theory, 157 
phase modulation (PM), 20, 108, 
729 

phase modulation schemes, 368 

phase nonlinearity, 127 

phase recovery, 450 

phase-recovery circuit, 345 

phase response, 719 

phase selectivity, 723-725 

phase sensitivity, 108 

phase-shift keying (PSK), 24 
and coherent systems, 490 
of phase modulation, 345 
represented by, 464 
signaling scheme, 344-345 

phase-shift keying (PSK) schemes, 
418 

phase spectrum, 715 

phase tree, 388 

phase trellis, 388-389 

phasors, 532 

photocathode, 4 

photodetector circuit, 58-59 

physical layer, 13 

W4-shifted DQPSK signals, 364 

W4-shifted DQPSK symbols, 363 

it/4-shifted QPSK scheme, 363 

W4-shifted QPSK signal 

demodulation of, 365 

residing in one of eight possible 
phase states, 362 
pictures 

and the human visual system, 4 
perception of, 4 
as a source of information, 4 
piecewise linear approximation, 
203 

Pierce, John R., 29 
pilot carrier, 99 

plain old telephone service (POTS), 

281-282 
plaintext, 742 
PM signal, 109 
PN sequence 
correlation properties of, 506 
as an independent and identically 
distributed (iid) binary 
sequence, 496 
as a reference signal, 550 



pointer, 581 

point-to-point communication, 2-3 
Poisson's sum formula, 718 
Poisson distribution, 59 
polar nonreturn-to-zero (NRZ) 

level encoder, 352 
polar nonreturn-to-zero (NRZ) 

signaling 
binary PCM system based on, 

253 

disadvantages of, 205-206 
polyvinylchloride (PVC) sheath, 16 
positive-going click, 150 
postdetection filter, 143 
power, available, 61 
power control 

in CDMA systems, 549 

use of, 549 
power gain, 523 

of an antenna, 520 

concept of, 521 

definition of, 520 
power-limited channel, 3 
power spectra, 347 
power spectral density, 44—46, 
347 

and amplitude spectrum, 50-52 

frequency portions of, 155 

graphical summary of, 75-76 

properties of, 46-47 

of random process, 50, 52 

significance of, 45 
power spectrum, 4, 45 
power theorem, 520 
Poynting vector, 519 
PPM. See pulse-position 

modulation 
precoded duobinary scheme, 

270-271 
prediction, 9 
prediction filter, 228 
pre-envelope 

basic property of, 732 

defined as, 725, 730 

determining, 726 

quadrature components of, 374 
prefix code 

definition of, 575-576 

distinguished by, 577 

property of, 576 
prefix coding, 575 
prefix condition, 575 
premodulation low-pass filter, 
396-397 
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preset threshold values, 277 
primary colors, 6 

represented by video signals, 6 

transmission of, 6 
primary rate, 214-215 
primitive BCH codes, 653 
primitive polynomial, 505 
principle of analysis by synthesis, 

551-552 
principle of rotational invariance 

illustration of, 330-331 

stated as, 330 
principle of superposition, 71 8 
principle of translational invariance 

application of, 331-332 

stated as, 331 
probabilistic code, 693 
probabilistic concepts, 703-707 
probabilistic decoder, 630 
probability 

axioms of, 704-706 

basic properties of, 705-706 

of bit error, 384 

of a correct decision, 357 

of error, 254, 409 

of symbol error, 258, 328-329, 
352 

probability, conditional, 706-707 
probability density function, 

67-68, 255, 594, 708-709, 

710 

probability distribution, 583 
probability of error, 328-329, 
497-499 

invariance of, 329-331 

for a noisy channel, 589 

union bound on the, 332-335 
probability of occurrence, 568 
probability of symbol error, 334 

determination of, 373 

evaluation of, 346-347 

formula for, 401 

for signal constellation, 337 
probability system, 704 
probability theory, 703, 705 
processing gain (PG) 

denned as, 229, 497 

produced by, 229 
product cipher, 749 
product modulator, 94, 98-99, 

111,490 
propagation, 4 
propagation effects, 532-535 
propagation time delay, 516 



protocol 

of the Internet, 13-14 

types of, 13-15 
pseudo-noise (PN) sequence, 480, 
488 

consists of, 288 
generation and properties of, 
480 

as a training sequence, 288 
pseudo-random-ordered sequence, 
500 

PSK. See phase-shift keying 
psychoacoustic modeling, 9 
psychovisual redundancy, 9 
public-key cryptographic system, 
755 

public-key cryptography, 742, 

755-757 
public-key system, 759 
public switched telephone network 
(PSTN), 237, 420 
as an analog network, 420, 421 
distortion on, 286-287 
efficient use of, 425 
pulse, 5 

pulse-amplitude modulated signal, 
429 

pulse-amplitude modulation 
(PAM), 20, 188-191, 236 
definition of, 188 
and modulator design, 277 
and natural sampling, 188 

pulse-code modulation (PCM), 
193, 615 
advantages of, 217, 237 
bandwidth requirement of, 218 
basic condition of, 194 
cost of advantages, 217-218 
definition of, 201 
as a form of digital pulse 

modulation, 237 
performance of, 227 
as the preferred method, 20 
for speech coding, 229-230 
use of, 217-218, 560 

pulse-code modulation (PCM) link, 
210 

pulse-code modulation (PCM) 

receiver, 258 
pulse-code modulation (PCM) 

signal, 208 
pulse-code modulation (PCM) 

system, 218 
basic operations of, 201 



characteristic of, 210 
description of, 201-209 
and interference, 210 
noise considerations in, 209-210 
operation of, 212 
performance influenced by noise, 
209 

pulse demodulator, 211 

pulse-duration modulation (PDM), 
20, 191-192, 236 

pulse-modulated signal, 237 

pulse modulation 
families of, 183 
forms of, 191-193, 237 
lossy nature of, 237-238 
method used to transmit, 211 
as source coding techniques, 
237 

standard digital form of, 20 
types of, 20 
pulse-modulation process 

incurred information loss of, 238 
loss of information and designer 
control, 238 
pulse modulation systems, 236 
pulse modulator, 211 
pulse noise jammer, 508 
pulse-position modulation (PPM), 

20, 192, 236-237 
pulse-position modulation (PPM) 
system 

versus frequency modulation 

system, 193 
noise analysis of, 193 
performance of, 193 
pulse shaping, 247 
pulse-shaping filter 

desirable properties of, 396-397 
Gaussian impulse response of, 
398 

pulse-shaping function, 373 
pulse-width modulation, 191 
punctured code, 676 
puncturing, 676 

Q 

QAM. See quadrature-amplitude 

modulation 
Q-channel, 97 

QPSK. See quadriphase-shift keying 
quadbits, 421 
quadratic receiver 

equation for, 405 

forms of, 405 
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quadrature-amplitude modulation 
(QAM), 97-98 
versus CAP, 369 
cross constellation, 371 
quadrature-amplitude modulation 
(QAM) constellations, 
369-370 
quadrature-amplitude modulators, 

433^34 
quadrature-carrier multiplexing, 

97-98, 354 
quadrature-carrier multiplexing 

system, 98 
quadrature channel, 408-409 
quadrature component, 93 
power spectral density of, 386 
properties of, 65-66 
role of, 93, 101 
quadrature modulation, 670 
quadrature noise component, 131 
quadrature null effect, 97 
quadrature-phase coherent 

detector, 97 
quadrature receiver 
channels of, 408 
using correlators, 405-406 
using matched filters, 405-406 
quadriphase-shift keying (QPSK), 
354, 354-361 
characterization of, 354-355 
error probability of, 356-358 
mode of operation, 425 
motivation for using, 508 
signal-space diagram, 354-355 
quadriphase-shift keying (QPSK) 

receiver, 360 
quadriphase-shift keying (QPSK) 
signal 

amplitude fluctuations, 362 
and binary PSK signal, 360 
commonly used constellations 

for, 362 
filtered, 361 

interference production, 396 

observations of, 360 

phase transitions of, 361 

power spectra of, 360-361 
quadriphase-shift keying (QPSK) 

transmitter, 359 
quality of service (QoS), 14 
quantization 

application of, 202-203 

and coding, 9 

purpose of, 8 

types of, 194-195 



use of, 195 
function of, 196 
types of, 220 

quantization noise, 195-197, 228 
designer's control of, 209 
in delta modulation, 221 
as a function of time, 195 
and human ear perception, 9 
in PCM systems, 209 

quantization process, 193-195, 
236 

in the generation of a binary 
PCM wave, 193 

illustration of, 195 

nonlinear nature of, 198 

results of, 20 
quantization table, 8 
quantized excitation, 552 
quantized filter parameters, 552 
quantizer 

characteristics of, 194 

classes of, 615 

components of, 199 

as a signal compressor, 615 

types of, 194 
quantizer, nonuniform, 202-203 
quantizer input, 221 
quantum, 194 
quaternary system 

eye diagram for, 294-295 
output of, 276 

R 

radiation efficiency factor, 520 
radiation intensity, 519 
radiation-intensity pattern, 520 
radio communication link, 522 
radio communication system, 3 1 
radio link analysis, 517-523 
radio propagation 

in free space, 512 

in urban areas, 532-533 
radix, 570, 576-577 
raised cosine spectrum 

flat portion of, 264-265 

rolloff portion of, 264-265 
RAKE receiver, 549-550 

basic idea of, 549 

consists of, 549 

as a diversity receiver, 549 

techniques of, 559 
random binary sequence, 482 
random experiment 

description of, 703 

features of, 703 



random hopping, 500 
random interference, 31-32 
random process 

average power of, 610 

classes of, 75 

definition of, 33 

ensemble averages of, 41 

expectations of, 41 

in linear systems, 42-44 

mathematical definition of, 
32-33 

parameter of, 75 

properties of, 32 

through a linear time-invariant 
filter, 42-44 
random variable, 33, 594, 708-710 

definition of, 708 

description of, 708 

distribution of, 55-56 

expected value of, 711 

mean of, 711 

standard deviation of, 712 

variance of, 712 
random vectors, 594 
raster scanning, 4-5 
rate distortion function, 612-613, 
616 

application of, 616 
definition of, 613 
rate distortion theory, 611-613 
application of, 612 
main parameters of, 613 
and Shannon's coding theorems, 
612 

Rayleigh's energy theorem 

definition of, 251 

use of, 251-252 
Rayleigh distribution, 68-69, 70, 
74-75 

Rayleigh fading channel, 536, 541 

binary signaling over, 542-547 

performance of, 545 
received signal 

components of, 31-32 

mean value of energy, 543 
received signal point, 323 
received vector, 660 
receive filter, 259 
receiver 

of an analog communication 
system, 88-89 

assumptions of, 403 

de-emphasis in, 154-155 

model, 130 

moving-coil, 15-16 
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noise performance of, 387 
as an optimum maximum 
likelihood detector, 436 
and preprocessing the received 
signal, 64 
receiver model, 130 
receiving antenna, 518 
reciprocity principle in antennas, 
521 

reconstruction filter, 187-188 

reconstruction levels, 194 

rectangular function, 262 

recursion theorem, 682 

recursive algorithm 

for phase recovery, 454 

for timing recovery, 457-458 

recursive symmetric convolutional 
(RSC) code, 675 

recursive Costas loop, 454 
convergence behavior of, 461, 
462 

operations of, 458 
phase-acquisition behavior of, 
459 

for phase synchronization, 454 
recursive early-late-delay 

synchronizer, 463-464 
redundancy 

addition of, 626 

basic forms of, 9 

controlled use of, 628 
redundant information, 227, 575 
Reed-Solomon codes, 654, 693 
Reeves, Alec, 27 
reference antenna 

definition of, 519 

as an isotropic source, 519 
reference signal, 557 
reflector antenna, 522 
regeneration, 208 
regenerative repeater, 208 
region of integration, 332 
regular-pulse excitation, 552 
regular turbo code, 692 
relative-frequency approach, 

703-704 
relative phase difference, 414 
relative phase shift, 532 
replication of delta function 

property, 717 
representation levels, 194 
reproduction quality, 5-6 
residual amplitude modulation, 112 
resolution, 6 

resolution of uncertainty, 568 



response time, 718 
reverse link 

versus forward link, 559 

subbands for, 547 
Rician distribution 

graphical presentation of, 70-71 

normalized form of, 71 
Rician fading channel, 536 
Rivest-Shamir-Adleman (RSA) 

algorithm, 759 
Rivest-Shamir-Adleman (RSA) 

system, 757 
rms duration, 722 
robust system, 22 
rolloff factor, 265 

frequency response for, 265-266 

time response for, 265-266 
root mean square (rms) bandwidth, 
721 

rotated noise vector, 330 
router 

denned as, 13 

and host devices, 10 

primary purpose of, 10 
row vectors, 633 
RS-232 standard, 6-7 
RSA algorithm, 757-758 
RSA cryptoalgorithm, 758 
RSA trapdoor one-way function, 
758 

S 

sample functions, 32-33 
sample point, 32, 704 
sample space, 32, 704 
sampling, 201-202 
sampling period, 184 
sampling process, 236 

and digital signal processing and 

digital communications, 184 
in the generation of a binary 

PCM wave, 193 
and pulse modulation systems, 

184, 236 
use of, 184 
sampling rate, 184, 201 
sampling theorem, 201, 236 
for band-limited signals, 

186-187 
derivation of, 186-187 
essence of, 184 
of a pulse-modulation system, 

186-187 
recurrent nonuniform equivalent 
form of, 427 



satellite 

for communication, 29 

in geostationary orbit, 18 

services of, 18-19 
satellite channel 

capabilities of, 516 

coverage of, 1 8 

remote area access, 18 
satellite communications, 514-517 

frequency band for, 19 

global coverage, 512 

most popular frequency band 
for, 515 

as a type of multiuser 
communications, 512 
satellite communication system, 18 

design of, 517 

global coverage, 559 

rely on, 512 

unique system capabilities of, 

18-19 
scalar quantization 
form of, 194 
use of, 194 
scalar quantizer 
conditions for optimality of, 

198-201 
designing of, 198 
as a simple signal compressor, 

615 

scanning, 4-5, 4-6 
scanning spot, 5 
scattered beams, 71 
scatterers, 71-72 
scattering function, 539 
Schwarz's inequality 

as a mathematical result, 
249-250 

proving, 313, 314 
SDMA, 516 

second flyback. See vertical 
retracesecond-order digital 
filter, 454-455 
second-order feedback system, 160 
secrecy, 745 
secret key 

versus public key, 759 

selection of, 745 
secret-key cryptoalgorithm, 751 
secret-key cryptography, 742-743 
secret-key system, 759 
secure channel, 743 
secure communications 

in a hostile environment, 479 

need for, 742 
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security of transmission, 490 
segments, 11 

See also packets 
separability theorem, 681 
sequential scanning 

of pictures, 4 

ptocess of, 4-6 
serial-to-parallel converter, 445 
Shannon, Claudecapacity theorem, 
611 

and "The Mathematical Theory 
of Communication", 27-28 

and the theoretical foundations 
of digital communications, 
27-28 

Shannon's capacity theorem, 611 
Shannon's fundamental bound for 

perfect security, 747 
Shannon's information capacity 

theorem, 23-24, 433 
Shannon's information theory, 617 
Shannon's second theotem, 616 
Shannon's third remarkable 

theorem, 616 
Shannon's thitd theorem, 599 
Shannon limit, 602 
Shannon model of cryptography 

method of confusion, 749 

method of diffusion, 749 

methods of designing, 749 
shift parameters, 531 
shift register, 481 
Shockley, William, 28 
shot noise, 58-60 
sideband, upper and lower, 91 
side information, 230 
sigma-delta modulation, 222 
signal 

definition of, 3-4 

detection in noise, 322-326 

dimensions of, 3-4 

received versus transmitted, 2 
signal bandwidth, 3 
signal constellation, 322-323, 337 

as circularly symmetric, 335 

constructed from one- 
dimensional PCM symbols, 
429 

defining minimum distance of, 
335 

signal detection problem, 322 
likelihood function for, 405 
stated as, 323 

signal energy-to-noise spectral 
density ratio, 252 



signal fading, 532-533 
signal-flow graph, 665-666 
signaling binary information, 345 
signaling interval, 568 
signaling rate, 276 
signal parameters, 403 
signal power average, 3 
signal regeneration, 208 
signal-space analysis, 337 
signal-space dimensionality, 3 12, 
493 

signal-space tepresentations 

of the interfering signal 
(jammer), 493 

of the transmitted signal, 493 
signals with unknown phase, 

403^06 
signal-to-mask ratio (SMR), 234 
signal-to-noise ratio 

basic definitions of, 3, 130-132 

at the device output, 524 

of an FMFB receiver, 153 

limitation of, 261 

of the soutce, 524 , 
signal-to-noise ratio gap, 432 
signal-to-quantization noise ratio, 
229 

signal transitions, 207 
signal transmission decoder, 326, 
349 

signal transmission encoder, 348, 
352 

signal variability, 530 

signal vector, 311 

simplex signals, 342 

signum function, 724 

sine function, 262 

sine wave plus narrowband noise, 

69, 69-71 
single-key cryptography, 742 
single keyed oscillator, 384 
single-letter distortion measure, 612 
single sideband (SSB) modulation, 
98-100, 163 
basic operation in, 103 
definition of, 93 
in frequency-division 
multiplexing, 106 
single-sideband modulated signal, 
98-99 

single-tone FM signal, 112-113 
single-tone jammer, 508 
single-tone modulation 

and a narrowband FM signal, 
110 



and a wideband FM signal, 110 
sinusoidal carrier wave 

defined as, 90 

waveform of, 490, 492 
sinusoidal modulating signal 

(wave), 110 
sinusoidal modulation, 112-113 
sinusoidal wave, 88 
slicing levels, 277 
slope circuit, 121-122 
slope network, 143 
slope overload distortion, 220, 
221 

slow FH/MFSK signal, 501 
slow FH/MFSK system, 502 
slow-frequency hopping, 500-502 
smoothness, 223 
SNR ratio. See signal-to-noise 

ratiosoft-decision coding, 630 
soft-decision decoding, 669 
soft decisions, 630 
soft input-hard output, 693 
soft input-soft output, 693 
SONET, 15 
source code, 574 
type of, 575 

variability in lengths of, 579 

soutce code word, 21 

source coding, 574 
dissection of, 616-617 
for efficient communication, ~ 
567 

with a fidelity criterion, 611-612 
source-coding theorem, 574-575, 

612, 616 
average code-word length of, 

611 

in Shannon's first theotem, 
574-575 
source decoder, 575 
source encoder, 21, 574 

functional requirements of, 574 

purpose of, 21 
spaced-frequency spaced-time 
correlation function, 538 
space diversity, 544-545 
space diversity technique, 546 
space-division multiple access 

(SDMA), 514 
space-time processor, 557 
spatial phenomenon, 534 
spatial sampling, 4-5 
spectral analysis, 110 
spectral content, 492, 493 
spectral decomposition, 443 
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spectrally efficient modulation, 347 

spectrally efficient schemes, 347 

spectral nulls, 368 

spectral shaping, 348 

spectrum, 3, 4, 715 

spectrum despreading 
in demodulation, 491 
as a linear operation, 492 

spectrum spreading 

as a linear operation, 492 
and phase modulation, 491 

speech coding 
applications of, 229-230 
design philosophy of, 230 
at low bit rates, 229-230 
techniques for, 551 

speech communication process, 4 

speech-production process, 4 

speech signal, 4 
as bipolar, 6 
limits of, 16 

sphere packing, 599-600 

split-phase signaling, 207 

splitter, 282 

spontaneous fluctuations, 58-61 
spreading code 

with pseudo-random properties, 
490 

use of, 490 
spread spectrum 

communications, 508 

important attribute of, 488 

notion of, 488-490 
spread-spectrum communication 
system 

advantage of, 479 

rejection of interference, 479 

requirements of, 493 
spread-spectrum modulation 

definition of, 479-480 

for military applications, 480 

principles of, 480 

to provide multipath rejection, 
480 

secure communications of, 479, 
480 

signaling techniques known as, 
479 

spread-spectrum techniques 
as direct-sequence spread 

spectrum, 490 
in passband transmission, 490 
versus standard modulation 
techniques, 480 
square constellations, 369-370 



square law, 193 
SSB modulated signal, 99 
SSB modulation, 134 
standard modulation techniques, 
480 

state diagram, 657-660, 659 
state probabilities, 681 
static picture, 4 
stationary process, 33-34 

versus strictly stationary, 33 

various names for, 36 
statistical average, 71 1-714 
statistical expectation operator, 
711 

statistical regularity, 703 
step-size, 194 

step-size parameter, 225-226 
stereo multiplexing 

in FM radio broadcasting, 124 

as a form of frequency-division 
multiplexing, 124 
stochastic process, 32 
stop-and-wait automatic repeat 

request, 628 
stop-and-wait strategy, 628 
stream ciphers, 744-746 

operation of, 744 

used in, 745 
strictly stationary process, 35 
Strowger, A. B., 27 
Strowger switch, 27 
subframes, 552 
subnets, 13 
substitution cipher 

description of, 749-750 

use of, 749-750 
successive errors, 232 
sufficient statistics, 321 
sum-product algorithm, 691 
Sunde's FSK, 381, 385-386, 388 
Sunde's FSK signal, 386 
superhet, 128 

superheterodyne receiver, 27, 
128-129 
consists of, 128 

differences between AM and FM, 
129 

survivor paths, 662 
switching center, mobile, 530 
symbol, 2 

symbol energy-to-noise spectral 
density ratio, 502 

symbol error 

average probability of, 276 
conditional probability of, 333 



symbol error probability 

versus bit error rate (BER), 
335-336 

to calculate, 357 

definition of, 209, 310 

evaluation of, 543 

formula for, 256-257 

to minimize, 310, 346 

as a ratio, 358 

simplification of, 335 
symbol rate, 501 
symbol shaping function 

defined as, 353 

energy spectral density of, 386, 
395 

symbol synchronization, 448 
symbol timing, 455, 458, 

463-464 
symbol timing recovery, 463-464, 

465 

symmetric modem configurations, 

421-425 
synchronization, 448-450, 493 
algorithmic (modern) approach, 

449 

basic modes of, 448 
classical approach to, 449 
implementation of, 449 
process of, 448 
as a statistical parameter 

estimation problem, 449 
in a TDM system, 212 
of transmitter and receiver 
clocks, 212 
synchronization problem 
approaches for solving, 449 
solution to, 493 
synchronizing pulses, 5, 212 
synchronous demodulation 

quadrature null effect, 96 
synchronous optical network 

(SONET), 15 
syndrome 

calculation of, 646-648 
importance of, 635 
properties of, 635—636 
syndrome calculator, 647 
syndrome decoding, 635-636, 638, 
639 

syndrome polynomial, 647 
synthesis equation, 442-443 
synthesis filter, 552 

consists of, 552 

as part of the encoder, 551 
systematic block code, 632 
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system capacity 

definition of, 549 

description of, 718 

to maximize, 549 
system-dependent scaling factor, 

132-133 
system design objectives, 3 
system-memory time, 718 

T 

TDMA. See time-division multiple 
access 

telecommunications environment, 

3-7 
telegraph, 26 

telegraphic code, Baudot's, 26 
telephone channel, 15-17, 287 
telephone circuit frequencies, 3 
telephone network, 27 

as a communication network, 
10 

first commercial service, 28 

primary purpose of, 15 
television network, 14 
television picture, 5-6 
television signals, 101-103 

modulation format of, 101-102 

as wideband signals, 7 
Telstar satellite, 29 
temporal autocorrelation function, 
284 

ternary code, 204 

terror-correcting RS code, 654 

theorem of irrelevance, 321, 
321-322 

theory of error-control codes, 485 

theory of spectral analysis of 
random processes, 46 

thermal noise, 32, 60 

Thevenin equivalent circuit, 60 

3-dB bandwidth, 721 

three-level output, 267-268 

threshold effect, 137-138 
AM and FM, 164 
clicks heard in, 149-150 
definition of, 138, 149 
in an envelope detector, 138 

threshold extension, 154 

threshold reduction, 152 

time-averaged autocorrelation 
function, 42, 51 

time-bandwidth product, 721-722 
choice of, 398 
as a design parameter, 397 



time compression (TC) 
multiplexing 

scheme, 278-279 

use of, 277-278 
time diversity, 544-545 
time-division multiple access 

(TDMA), 211, 211-212, 513 

efficient system, 516 

as wireless communications 
systems, 547-550 
time-division multiplexing (TDM) 

concept of, 211 

defined as, 21, 105 

use of, 211-212 
time-domain description, 720 
time-flat channel, 542 
time-frequency mapping, 9 
time response, 266 
time-scaling property, 722 
time slot, 513 
time-to-frequency mapping 

network, 235 
time-varying phasqr, 728 
time-varying transfer function, 536 
timing error, 264 
timing synchronization, 455 
Toeplitz property, 225 
toll connection, 16 
Tomlinson-Harashima precoding, 
430 

tone-modulation analysis, 119 
tracking, 493 
tracking filter, 154 
trade-offs, 602 
training mode, 290 
transceiver, 278 
transistor, 28 
transition matrix, 582 
transition probability, 582, 616 
transmission bandwidth 

defined as, 118 

instantaneous spreading, 499 
transmission delay, 260 
transmission path, 212 
transmission security, 490 
transmit filter, 259 
transmitted code vector, 660 
transmitted FH/MFSK signal, 500 
transmitted power 

definition of, 3 

primary communication 
resource, 92 
transmitted pulse amplitude, 253 
transmitted pulse shape, 261 



transmitted signal, 280-281 
transmitted signal energy per bit, 
258 

transmitted signal point, 322-323 
transmitted TV signal, 102 
transmitter 

of an analog communication 
system, 88-89 

by combining operations, 414 

location of, 2 

power limited, 597 

purpose of, 2 

use of pre-emphasis in, 154-155 
transmitting antenna 

function of, 518 

mounting of, 17-18 

as a point source, 519 

and power density, 521 
transorthogonal signals. See 

simplex signals 
transponder, 19, 514-515 
transponder channel, 515 
transposition cipher 

description of, 750 

use of, 750 
transversal equalizer, 286 
trapdoor one-way function, 755 
traveling-wave tube amplifier, 
516 

trellis, 657-660, 661-662 
trellis-coded modulation, 668-669 
trellis codes 

for band-limited channels, 
668-669 

design of, 669 
trellis coding 

as an error-control coding 
technique, 430 

as a forward-error correction 
scheme, 424 
trellis encoder, 425 
turbo codes 

consist of, 674 

development of, 674 

performance of, 676-677 

properties of, 682, 693 

termination approaches of, 676 
turbo coding, 674 
turbo decoder 

basic structure of, 677-678 

complexity of, 684 

details of, 683 
turbo decoding, 677-680, 682-683 
turbo encoder, 683 
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twisted-pair cable 

versus coaxial cables, 17 

consists of, 16 

susceptible to, 16-17 

uses of, 277 
2B1Q code, 281 

amplitude levels of, 317 

compared to other line codes, 
281 

desirable properties of, 281 
as the North American standard, 
281 

two-dimensional matched filter, 

378-379 
two-dimensional optimum receiver, 

378 

two-key cryptography, 742 
two-stage spectral analysis, 110 
two-step subspace procedure, 
556 

two-way transmission, 106 
U 

Ungerboeck, G., 28 
Ungerboeck codes, 670 

for 8-PSK, 670-672 

asymptotic coding gain of, 673 
unicity distance, 748 
uniform quantizer, 196 
union bound, 337 

illustration of, 333 

simplification of, 335 

as a useful upper bound, 
332-333 

use of, 401 
union of events, 333 
unipolar nonreturn-to-zero (NRZ) 

signaling, 205 
unipolar return-to-zero (RZ) 
signaling 

disadvantages of, 207 

feature of, 207 
uniquely decodable, 574 
unitary matrix, 443 
unit delay, 219 
unit-delay elements, 646 
universal curve, 118-119 
unmodulated carrier, 108 
up conversion, 104-105 
upconverter, 448 
uplink, 19, 514-515 
upper bound, 401 
upstream data transmission, 
281-282 



V 

V.32 modemand nonredundant 
coding, 423^124 
phase changes in, 422—423 
switching to QPSK mode, 424 
and trellis coding, 423^124, 425 

V.32 modem standardand 
alternative modulation 
schemes, 421 
characteristics of, 421 

V.90 modem, 431 

van Duuren, H, C. A., 28 

Van Vleck, J. H., 27 

variable-length code, 574 

variable nodes, 685 

variance, 579 

VDSL. See very-high-rate digital 

subscriber lines 
vector product, 681 
vector quantizer 
advantage of, 615 
encoding process in, 615 
versus scalar quantizer, 615 
signal-to-quantization noise ratio 
for, 615 
vectors, 633 
vector space, 554-555 
Vernam cipher, 747 
very-high-rate digital subscriber 
lines (VDSL) 
advantages of, 446 
use of, 446 
very-large-scale integrated (VLSI) 
circuits 
development of, 28 
vestigial sideband (VSB) filter 
frequency response of, 102 
magnitude response of, 100-101 
vestigial sideband (VSB) modulated 
wave 

methods of generating, 100 
quadrature component of, 
102-103 
vestigial sideband (VSB) 

modulation, 100-101, 163 
vestigial sideband (VSB) shaping 

filter, 102 
vestigial sideband modulation 
definition of, 93 
and its role in commercial TV 
broadcasting, 101-102 
video bandwidth, 6 
video-on-demand, 9, 282 
video signal, 4-5, 6 



virtual communication, 13 
Viterbi algorithm, 661-663, 670 
difficulty in the application of, 
662 

as a maximum likelihood 

decoder, 662 
as a maximum likelihood 
sequence estimator, 662 
VLSI. See very-large-scale 

integrated (VLSI) circuits 
vocal tract, 4 
voiceband modem, 420 

versus digital subscriber lines, 
446 

operational environment of, 447 
voice communication, 22 
voice effect, Donald Duck, 99-100 
voice signals, 99 
voice spectrum, 3 
voltage-controlled oscillator 

(VCO), 152-153, 157-158 
von Neumann, John, 28 
VSB. See vestigial sideband 

W 

water-filling interpretation, 
610-611 

water-filling solution, 438 

waveform, 21, 276 

of important line codes, 204-207 
in modulation, 490, 492 

waveform distortion, 102-103 

wavelength-division multiplexing 
(WDM), 21 

wave motion, 18 

weak signal suppression, 142 

weight vector, 554 

white Gaussian noise, 62 
identically distributed, 334 
process, 392 

white noise, 61-63 

autocorrelation function of, 

61-62 
characteristics of, 61-62 
mathematical properties of, 62 
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CHAPTF.R 1 



Problem 1 1 

As an illustration, three particular sample functions of the random process X(t) , 
corresponding to F = W/4, W/2, and W, are plotted below: 

sin(2irWt) 





W 

sin(2ir — t) 
4 



2 
W 



2 
W 



To show that X(t) is nonstationar y, we need only observe that every waveform illustrated 
above is zero at t = 0, positive for 0 < t < 1/2W, and negative for -1 /2W < t < 0. Thus, 
the probability density function of the random variable XCt^ obtained by sampling X(t) at 
t 1 = 1/4W is identically zero for negative argument, whereas the probability density 
function of the random variable X(t 2 ) obtained by sampling X(t) at t = -1 /4W is nonzero 
only for negative arguments. Clearly, therefore, 

f X(t 1 )^ x ^ f X(t )^ x 2^> and the random Process X(t) is nonstationar y. 



1 



Problem 1.2 



X(t) = A cos(2Tif t) 
c 



Therefore , 



X. = A COS(2TTf t. ) 

i c 1 

Since the amplitude A is uniformly distributed, we may write 

1 



cos(2rrf t.)* 
c 1 



fy (X.) = 

x i 1 / o, 



0 < x < cos(2irf t.) 
— i — c 1 



other wi se 



cos (2irf t. ) 

c 1 



cos (2irf t . ) 

c 1 



Similarly, we may write 



X i+T = A cos[2 1 rf c (t i +x)] 



and 



cos[2Trf (t.+t)]' 

C 1 



0, 



0 < x < cos[2irf (t.+x)] 
- 2 — c l 



otherwise 



We thus see that f x (x j ) i - (x ) , and so the process X(t) is nonstationary . 



l+T 



Problem 1.3 

(a) The integrator output at time t is 



Y(t) = / X(t) dx 
0 



= A / cos(2irf x) dx 
0 C 



A sin(2irf t) 



2nf c — c 



Therefore , 



sin(27rf t) 

E [Y (t) ] = C E [A ] = 0 

c 

sin^(2Trf t) 

VarCY(t)] = ^— Var[A] 

(2nf Q ) 2 

sin 2 (2Trf t) 

= 2 °A (1 > 

(2rrf ) d A 
c 

Y(t> is Gaussian-distributed, and so we may express its probability density function as 

✓ 27 f 2rr 2 f 2 

f Y(t) (y) = q. sinC^f t) eXp[ ~ . 2 fo ° „ 2 y J 
A c sin (2irf t) a. 

c A 

(b) From Eq . (1) we note that the variance of Y(t) depends on time t, and so Y(t) is 
nonstationar y. 

(c) For a random process to be ergodic it has to be stationary. Since Y(t) is 
nonstationary , it follows that it is not ergodic. 

Problem 1.4 

(a) The expected value of Z(t^) is 

E[Z(t.,)] = cos(2irt 1 ) E[X] + sin(2irt.,) E[Y] 
Since E[X] = E [Y ] = 0, we deduce that 

ECZCt^] = 0 
Similarly, we find that 

E[Z(t 2 )3 = 0 
Next, we note that 

Cov[Z(t 1 )Z(t 2 )] = E[Z(t 1 )Z(t 2 )] 

= E{[X cos^tt^) + Y sin(2nt 1 )][X cos(2irt 2 ) + Y sin(2irt 2 >]} 
= cosCart^ cos(2irt 2 ) E[X 2 ] 



+ [ co s( 2irt 1 ) sin ( 2irt 2 )+ sin ( 2irt 1 ) cos( 2nt 2 ) ]E [XY ] 



+ sin(2nt 1 )sin(2nt 2 )EtY 2 ] 



Noting that 

E[X 2 ] = 0 2 + {E[X]} 2 = 1 

E[Y 2 ] = a 2 + CE[Y]} 2 = 1 

E[XY] = 0 
we obtain 

Cov[Z(t 1 )Z(t 2 )] = cos(2Trt 1 )cos(2 1 rt 2 )+sin(2Trt l )sin(2Trt 2 ) 

= cos[2 l r(t 1 -t 5 )] (1) 
' of -fcke ^roco^S 

Since every weighted sum of the samples^ZCt) is Gaussian, it follows that Z(t) is a 
Gaussian process. Furthermore, we note that 

°z 2 ( tl ) = Etz2(t i» = 1 

This result is obtained by putting t^t 2 in Eq. (1). Similarly, 
°Z 2 (t 2 ) =™ 2 «2» - 1 

Therefore, the correlation coefficient of Z(t^ and Z(t 2 ) is 
Cov[Z(t 1 )Z(t 2 )] 
P °Z(t 1 )°Z(t 2 ) 

= cos[2ir(t 1 -t 2 )] 

Hence, the joint probability density function of Z(t.,) and Z(t 2 > 
f Z( tl ),Z(t 2 ) (z r z 2> = C exp[-Q(z 1t z 2 )] 

where 



C = 



,/C- 2 ' 



2ir/1 -cos [2 l r(t 1 -t 2 )] 



2ir sin[2 1 r(t 1 -t 2 )] 



12 2 
Q(z 1t z 2 ) = 2" {z^ - 2 cos[2tt (t 1 -t 2 )]z 1 z 2 + z~) 

2 sin [2ir (t^tj)] 



4 



(b) We note that the covariance of Z(t^) and Z(t 2 ) depends only on the time difference 
t -j — 1 2 • The process Z(t) is therefore wide— sense stationary. Since it is Gaussian it is 
also strictly stationary. 

Problem 1.5 
(A>) Let 

X(t) = A + Y(t) 

where A is a constant and Y(t) is a zero-mean random process. The autocorrelation 
function of X(t) is 

R x (x) = E[X(t+x) X(t)] 

= E{[A + Y ( t+x ) ] [A + Y(t)]} 

= E[A 2 + A Y(t+t ) + A Y(t) + Y(t+x) Y(t)] 

= A 2 + R y ( T ) 

o 

which shows that R x (x) contains a constant component equal to A . 
(V) Let 

X(t) = A cos(2irf t + 6) + Z(t) 

c c 

where A c cos(2irf c t+6 ) represents the sinusoidal component of X(t) and 6 is a random phase 
variable. The autocorrelation function of X(t) is 
R x (t) = E[X(t+x) X(t)] 

= E{[A cos(2wf t + 2irf x + 6) + Z(t+x)] [A cos(2iff t + 6) + Z(t)]} 
c c c c c 

2 

= E [A cos(2irf t + 2nf x + 6) cos(2uf t + 0)] 
c c c c 

+ E[Z(t+x) A cos(2irf t + 9)] 
c c 

+ E[A cos(2Trf t + 2irf x + 6) Z(t)] 
c c c 

+ E[Z(t+x) Z(t)] 

= (A 2 /2) cos(2Tif x) + R_(x) 
c c Z 

which shows that R x (t) contains a sinusoidal component of the same frequency as X(t) . 
Problem 1.6 

(a) We note that the distribution function of X(t) is 



5 



F x(t) (x) = 



o, 

2' 
1. 



x < 0 

0 < x < A 
A < x 



and the corresponding probability density function is 
f x(t) (x) = •! 6(x) + ^ 6(x - A) 



which are illustrated below: 



1.0 



fcx) 

*C4> 



0 A 

(b) By ensemble-averaging, we have 



E[X(t)] = / x f x(t) (x) dx 



/ x [1 6(x) + j 6(x - A)] dx 



A 
7 



The autocorrelation function of X(t) is 

R x (t) = E[X(t+x) X(t)] 

Define the square function Sq T (t) as the square-wave shown below: 

0 



6 



S <5m (t) 




Then, we may write 

R Y (x) = E [A Sq T (t - t. + t) • A Sq T (t - t.)] 
* 0 d 0 a 

2 " 

= A / Sq T (t - t d + t) Sq T (t - t d ) f T (t d > dt d 
-oo 0 0 d 

T /2 

= A 2 / ° Sq T (t - t . + x ) Sq„ (t - t .) • ^- dt 
-T Q /2 T 0 d T 0 d T 0 d 

Since the wave is periodic with period T Q , R x (x) must also be periodic with period T Q . 
(c) On a time-averaging basis, we note by inspection of Fig. P/.^that the mean is 

<x(t)> = | 

Next, the autocorrelation function 

T /2 

<x(t+x)x(t)> = / ° x(t+x) x(t) dt 
0 -T Q /2 

2 

has its maximum value of A /2 at x = 0, and decreases linearly to zero at x = Tq/2. 
Therefore , 

2 T 
<x(t+x) x(t)> = 5- (1 - 2 -^), fx | < . 
* 0 
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Again, the autocorrelation must be periodic with period T 



(d) We note that the ensemble-averaging and time-averaging procedures yield the same set 
of results for the mean and autocorrelation functions. Therefore, X(t) is ergodic in both 
the mean and the autocorrelation function. Since ergodicity implies wide-sense 
stationarity, it follows that X(t) must be wide-sense stationary. 

Problem 1 .7 

(a) For )t\ > T, the random variables X(t) and X(t+x) occur in different pulse intervals 
and are therefore independent. Thus, 

E[X(t) X(t+x)] = E[X(t)] E[X(t+x)], It | > T. 

Since both amplitudes are equally likely, we have E[X(t)] = E[x(t+t)] = A/2. Therefore, 
for It I > T, 

A 2 

. R x (t) = ±- . 

For |t | £ T, the random variables occur in the same pulse interval if t . < T - jx | . If 
they do occur in the same pulse interval, 

2 

E[X(t) X(t+x)] = \ A 2 + 1 0 2 = j- . 

We thus have a conditional expectation: 

E[X(t) X(t+T)] = A 2 /2, t d < T - |t | 

2 

= A /4, otherwise . 
Averaging over t^, we get 

T-fxl A 2 T 2 

R Y (x) = / # dt . + / tj=t dt. 

X 0 21 d T-ixl 4T d 

2 2 

= t" (1 " i T 1) + TT ' 1x1 i T 

(b) The power spectral density is the Fourier transform of the autocorrelation function. 
The Fourier transform of 

g(T) = 1 — ~- , St I _< T 

= 0 , otherwise, 
is txtj 

G(f) = T sinc 2 (fT). g 



Therefore, 



2 ^ 



S x (f) = 5- 6(f) + sinc 2 (fT) 



We next note that 



A 2 00 A 2 
^-S 6(f) df = , 

—AO 

2 00 p 

A 2 A 

Tsinc^fT) df = J- , 

A 2 

/ S x (f) df ± R x (0) = j- . 

■00 

It follows therefore that half the power is in the dc component. 

Problem 1.8 
Since 

Y(t) = g (t) + X(t) + /3/2 
and gp(t) and X(t) are uncorrelated, f^evi 
C y (t) = C ( T ) +C ( T ) 

where V gp (T) is the autocovariance of the periodic component and ^ x (t) is the 
autocovariance of the random component. C^( x ) i s the plot in figure Pi. 8 shifted down by 
3/2, removing the dc component. Q__(t) and C ( T ) are plotted below: 

gP A 
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Both g p (t) and X(t) have zero mean, 

(a) The power of the periodic component g (t) is therefore, 

A P 

1 T 0 /2 2 /"> 1 

=" / g*(t) dt =C (0) = 1 

T 0 -T 0 /2 P Sp 2 

ewe raj e 

(b) The power of the random component X(t) is 
E[X 2 (t)] = C X (0) = 1 

Problem 1.9 

(a) R^OO = E[X(t+x) Y(t)] 

Replacing x with -x: jq 
R^C-x) = E[X(t-x) Y(t)] 



Next, replacing t-x with t, we get 
%(-!) = E [Y ( t+T ) X(t)] 
= R yx (t) 

(b) Form the non-negative quantity 

E[{X(t+t) + Y(t)} 2 ] = E[X 2 (t+x) + 2X(t+x) Y(t) + Y 2 (t)] 

= E[X 2 (t+x)] + 2E[X(t+x) Y(t)] + E[Y 2 (t)] 
= R x (0) + 2R xy (x) + R y (0) 



Hence , 



R x (0) + ZR^Cx) + R y (0) > 0 



or 



1R XY (x)l i I t R x (0) + R y (0)] 

Problem 1.10 

(a) The cascade connection of the two filters is equivalent to a filter with impulse 
response 

oo 

h(t) = / h^u) h 2 (t-u) du 

—00 

The autocorrelation function of Y(t) is given by 



R y (x) = / / h(x^ h(x 2 ) R x (x - x 1 +x 2 ) dx 1 dx 2 



(b) The cross-correlation function of V(t) and Y(t) is 

RyyCT) = E[V(t+X) Y(t)] - 

The Y(t) and V(t+x) are related by 



Y(t) = / V(X) h 2 (t-X) dX 



Therefore , 



RyyCx) = E[V(t+x) / V(X) h 2 (t-X) dX] 
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= ; h 2 (t-X) E[V(t+r) V(X)] dX 



= / h 2 (t-X) R v (t+x-X) dX 



Substituting X for t-X: 



RyyCx) = / h 2 (X) R y (t+X) dX 



The autocorrelation function Ry(x) is related to the given R x (t) by 

CO oo 

R v (x) = f f h^Cx.,) h^tg) R x (i-r^-n 2 ) dx.,dx 2 



Problem 1.11 

(a) The cross-correlation function Ryx^ T ^ is 

R yx (x) = E [Y ( t+x ) X(t)] 
The Y(t) and X(t) are related by 

OO 

Y(t) = / X(u) h(t-u) du 

.*00 

Therefore, 

oo 

R YX (x) = E[ / X(u)X(t) h(t+x-u) du] 

— oo 

00 

= / h(t+x-u) E[XCu)X(t)] du 

00 

= / h(t+ T -u) R x (u-t) du 

■^o 

Replacing t+x-u by u: 



R YX (x) = / h(u) R x (x-u) du 



(b) Since R^^) = R Y x^~ T ^» we have 

12 



—•□0 

Since R x (t) is an even function of x: 

oe 

R xy (x) = / h(u) R x (t+u) du 

— oo 

Replacing u by -u: 

00 

RyyCx) = / h(-u) R x (x-u) du 

_00 

Cc) If X(t) is a white noise process with zero mean and power spectral density N Q /2, we 
may write 

N 

R x (x) = ^-6(x) 
Therefore, 

N 00 



R yx (x) = y-/ h(u) 6(x-u) du 

-co 

Using the sifting property of the delta function: 



N 0 

R YX (x) = ^- h(x) 

That is, 

h(x) = §- R YY (x) 
N 0 YX 

This means that we may measure the impulse response of the filter by applying a white 
noise of A spectral density N Q /2 to the filter input, cross-correlating the filter output 
with the input, and then multiplying the result by 2/Nq. 

Problem 1.17. 

(a) The power spectral density consists of two components: 

(1) A delta function 6(t) at the origin, whose inverse Fourier transform is one. 

(2) A triangular component of unit amplitude and width 2f Q , centered at the origin; 
the inverse Fourier transform of this component is f Q sine (f Q x). 

Therefore, the autocorrelation function of X(t) is 

13 



R x (t) = 1 + f Q sinc 2 (f 0 x ) 
which is sketched below: 







R x (t) 


<™ 1+f o 






1 

i 


1 


1 " 7 
1 


1 

1 


1 
1 
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1 

1 
1 


3 


2 


-2_ ( 


) 1_ 


2 


3 


" f o 


" f o 


f o 




f o 





(b) Since R^(x ) contains a constant component of amplitude 1, it follows that the dc 
power contained in X(t) is 1. 

(c) The mean-square value of X(t) is given by 
E[X 2 (t)] = R x (0) 

= 1 + f 0 

The ac power contained in X(f) is therefore equal to f Q . 

(d) If the sampling rate is f Q /n, where n is an integer, the samples are uncorrelated. 
They are not, however, statistically independent. They would be statistically independent 
if X(t) were a Gaussian process. 

Problem 1 n 

The autocorrelation function of n 2 (t) is 
■ R N (t r t 2 ) = E[n 2 (t.,) n 2 (t 2 )] 

= EUn.Ct..) cos(2irf t 1 + 0) - n^t.) sin(2nf t )] 
II c \ II ci 

• [n 1 (t 2 ) cos(2irf c t 2 + 0) - n^tg) sin(27rf,t 2 + 0)]} 
= ECn.Ct.) n.Ct-,) cos(2irf t 1 + 0) cos(2irf t 0 + 0) 

II \ d C I C d 

- r\At.) n.(t 0 ) cos(2irf t.. + 0) sin(2rrf t 0 + 0) 

\ \ i d cl c d 

- n^(t^) n^tg) sin(2irf c t 1 +0) cos(2nf t 2 + 0 ) 
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+ n-jCt.,) n.,(t 2 ) sin(2irf c t 1 + 0) sin(2Trf c t 2 + 0 )] 
= Efn-jU.,) n^tg) cos[2jrf c (t.,-t 2 )] 

- n^(t^) n.,(t 2 ) sin[2Trf c (t 1 +t 2 ) + 20]} 
= ECn^t^ n 1 (t 2 )] cosL2nf Q it^t 2 n 

- Etn^t^ n 1 (t 2 )] • E{sin[2irf c (t 1 +t 2 ) + 20]} 

Since 0 is a uniformly distributed random variable, the second term is zero, giving 
R N 2 (t 1' t 2 ) = R N 1 (t 1' t 2 ) cos C2irf c (t 1 -t 2 )] 

Since n^t) is stationary, we find that in terms of t = t.,-t 2 : 

R N (x) = R N (x) cos(2irf t) 
. 2 1 c 

Taking the Fourier transforms of both sides of this relation: 



s N Jf) =^Cs Ni (f + f c ) +s Hi (f-f o >] 



With S N (f) as defined in Fig. ?t„ t J, we find that S N (f) is as shown below: 
1 2 




Problem 1.14 

The power spectral density of the random telegraph wave is 



S x (f) =/ R x (x) exp(-j2irfx) dr 



0 

/ exp(2vx) exp(-j2irfT) dr 



+ / exp(-2vx) exp(-jairf T ) dr 
0 



1 0 



1 1 
2(v-jirf) + 2(v+jirf) 



2 2 2 
v +ir f 



The transfer function of the filter is 
1 



H(f) = 



1 + j2irfRC 



Therefore, the power spectral density of the filter output is 
S y (f) = fH(f) I 2 S x (f) 



[1 + (2TrfRC) 2 ](v 2 -h; 2 f 2 ) 



To determine the autocorrelation function of the filter output, we first expand Sy(f) 
in partial fractions as follows 

Sy(f) = ?22 ^ 2 2~~2 + ~2 2 2"* 

1 - UR C v (1/2RC) + w^f 42 v d + tt f 

Recognizing that 

15 



exp(^2v«t()^ V 



v- 2 ^ 2,2 



exp(-Hl/RC)-± 1/2RC 



(1/2RC) 2 -hr 2 f 2 



we obtain the desired result: 



R Y (x ) = \ [- exp(-2o |x | ) - 2RC exp(- 

1 1-HR^Cv V 
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Problem 1.15 
We are given 

y(t) = ( x(T)dx 

J t-T 

For x(t) = 8(0, the impulse response of this running integrator is, by definition, 
h{t) = f 5(t)Jt 

J t-T 

= 1 for t-T<Q<t or, equivalently, 0<t<T 
Correspondingly, the frequency response of the running integrator is 

H(f) = f h(t)exp(-j2nft)dt 

•* —CO 

T 

= f exp(-j2nft)dt 
J o 

= ]Wt [1 - eXP( - j2nfr)] 

= rsinc(/T)exp(-;7i/T) 

Hence the power spectral density Sy(f) is defined in terms of the power spectral density S x (f) as 
follows 

S Y (f) = \H(f)\ 2 S x (f) 

= r 2 sinc 2 (/T)S x (/) 

Problem 1.16 

We are given a filter with the impulse response 




aexp(-at), 0<t<T 
0, otherwise 
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The frequency response of the filter is therefore 



H(f) = f h(t)cx V (-j2nft)dt 

J —oo 

C T 

= aexp(-at)exp(-j2nft)dt 
J 0 

c t 

= a] exp(-(a + j2nf)t)dt 
J o 



U [-exp(-(a + j2nf)t]l 



a + j2nf 



a + j2nf 



[l-ex V (-(a + j2nf)T] 



a [l-e at (cos(2nfT)-jsm(2nfT)X 



a + j2nf 
The squared magnitude response is 



\H(ff = 



a (l-e aT cos(2nJT)) 2 + (e aT sm(2nfT)) 2 



2 . 2 r 2 

a +47t / 



a [l-2e aT cos(2nJT) + e 2aT (cos 2 (2nfT) + sin 2 (27t/T))] 



2 . 2 r 2 

a +471 / 



a no i^~ct-\ , -2aT-. 

[l-2e cos(2tc/T) + <? ] 



2 . 2 .2 

a +4n f 



Correspondingly, we may write 



S Y (f) = 2 a 2 2 [l-2 g - ar cos(27t/T) + g " 2ar ]5 x (/) 
a +4tc / 
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Problem 1.17 

The autocorrelation function of X(t) is 
R x (t) = E[X(t+x ) X(t)] 

= A 2 E[cos(2TrFt + 2ttF t -e) cos^Ft -0)] 
A 2 

= 2~ E[cos(4TrFt + 2ttF t -r 20 ) + cos(2ttF t )] 
Averaging over e, and noting that 0 is uniformly distributed over 2rr radians, we get 
A 2 

R x (t) = 2~ E[cos(2ttFt)] 
A 2 " 

= 2~ / f F (f) cos(2irfT) df (D 

-00 

Next, we note that R x (t ) is related to the power spectral density by 

R x (t) =/ S x (f) cos(2irfT) df ( 2 ) 

-co 

Therefore, comparing Eqs . (1) and (2), we deduce that the^ spectral density of X(t) is 

S X (f) = |" f F (f) 
When the frequency assumes a constant value, f (say), we have 

*V f) = 5- 6(f-f„> + J- 6(f+f ) 
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, correspondingly, 



.2 2 
S x (f) = — 6(f-f c ) + J- 6 (f+f c ) 



Problem 1.18 



Let o^ 2 denote the variance of the random variable X k obtained by observing the random process 
X(t) at time 1^. The variance o^ 2 is related to the mean-square value of X k as follows 



4 = E[X k 2 ] - u| 



where^s* x = E[X k ]. Since the process X(t) has zero mean, it follows that 



o x = E[X^] 



Next we note that 



E[X 2 ] = J_~ S x (f)df 



We may therefore define the variance Oj^ 2 as the total area under the power spectral density S^f) 
as 



4 = I~ S x(f)df 



(1) 



Thus with the mean u x = 0 and the variance o x 2 defined by Eq. (1), we may express the probability 
density function of X k as follows 



fx k 00 - 



V25Fa x 



exp 



2at 
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Problem 1.19 

The input-output relation of a full-wave rectifier is defined by 

X(t. ), X(t ) > 0 

Y(t. ) = |X(t.)| 

' -x(t k ), X(t k ) < 0 

The probability density function of the random variable X(t k ) , obtained by observing the 

input random process at time t^, is defined by 

1 x 2 

f xct„> (x> = -=~ ex P<- Tt> 

To find the probability density function of the random variable Y(t k ), obtained by 
observing the output random process, we need an expression for the inverse relation 
defining X(t k ) in terms of Y(t R ). We note that a given value of Y(t k ) corresponds to 2 
values of X(t k ) , of equal magnitude and opposite sign. We may therefore write 

X(t k ) = -Y(t k ), X(t k ) < 0 
X(t k ) = Y(t k ), X(t k ) > 0 . 
In both cases, we have 

The probability density function of Y(t k ) is therefore given by 

) dX(t k ) | ) dX(t. ) 

f Y(t k > ( *> = f xa k ) (x = i dYcy [ + f x( V (x = y> • hm^ 



■A 



-i„p(-^) 

rr a 2( 2 



We may therefore write 



f Y(t k ) ( *> 



A a exp( - rr 5 ' y > 0 

do 



0 , y < 0 . 
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which is illustrated below: 



/ (3) 








U • / Z7 D 




a 




0 


Problem 1.20 



(a) The probability density function of the random variable Y(t k ), obtained by observing 
the rectifier output Y(t) at time t k , is 



/TSy 



ex 



y > 0 



f Y(t k ><y> *' 

where = E[X 2 (t k )] - {E[X(t k >]} 2 
= E[X 2 (t k )] 

= R X (0) 

The mean value of Y(t k ) is therefore 



y < 0 



E[Y(t k )] = / y f (y) dy 

- 00 k . 



/ /y exp(- 2-—) dy 



/2n 0 



2o: 



(1) 



Put 



Then, we may rewrite Eq. (1) as 



_ oo 2 

ECY(t k )] = /I °x ' u2 ex P ( " T"> du 

2 

= °X 
= R x (0) 

(b) The autocorrelation function of Y(t) is 

R y (t) = E [Y( t+ t) Y(t)] 
Since Y(t) = X 2 (t), we have 

R v (t) = E[X 2 (t+x) X 2 (t)] 



oo CO _ _ 

2 2 



1 1 X 1 X 2 f X(t, + x ),X(t, ) (X 1' X 2 ) dx 1 dx 2 < 2 > 

— oo —oo K K 



The X(t k -Kr ) and X(t k ) are jointly Gaussian with a joint probability density function 
defined by 

1 x, - 2p (t) x. + x 0 

hit +T ),x(t ) (x r x 2> = ex Pt -2— -g— 3 

k k 2, a 2 /l-p 2 (x) 2a x (1-p x (x)) 

2 

where a x = R x (0) , 

Cov[X(t k +x)X(t k )] 

P y (t ) = 



T 1 ' " 2 



°X 



R x (T) 
R x (0) 



Rewrite Eq. (2) in the form: 



00 



2 

2 , X 2 



R yfr) = / x 2 exp( 8^ x 2 ) dx 2 (3) 

3r a 2 /l-p|(T) "" ^X 



where 

00 P Cx- - p y (t ) x 9 ] 2 

g(x 2 ) = / x* exp{ L _ } dx, 

— 2o x [1 -p x (x)] 1 
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Let 



u _ *1 " P X (T) x 2 



/ 2 

Then, we may express g(x 2 ) in the form 

r— — — oo 2 

g(x 2 ) = a x / 1-p x (t) / expC- y~) {p 2 ( T ) x 2 + c 2 [1 - P 2 (t)]u 2 + 2a p /l-p 2 (x) ux 2 l du 



However , we note that 

2 

/ exp(- — ) du = /2ir 



oo 2 

u 

/ u exp(- -5—) du = 0 



00 2 
/ u 2 exp(- y~) du = /2lr 



Hence , 



/ 2 2 2 2 2 

g(x 2 ) = Ox ^TrCl-pjjd)] {px(t) x 2 + [1 - ^(t)]} 

Thus, from Eq. (3): 

2 

00 Y 

1 2 2 2 2 2 2 

. Ry(x) = — / x 2 exp(- — {^(t) x 2 + H-p^x)]} dx 2 

/2tt ©x -» 2a x 
Using the results: 

2 

00 _ Y 

2 2 > — 
/ x 2 exp( dx 2 = /2H ^ 

2 °X 

2 

* 4 X 2 — 5 

/ x 2 exp( 2) dx 2 = 3/2tt 0^ 

2o X 
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we obtain , 



R y (T) = Px (T) + a x [1 " P X (T)] 
= °X [1 + 2 Px (T)] 

p 

Since a x = R x (0) 



R x (t) 
p X (T) = R^O) 



we obtain 



5 R5(T) 

R Y (x) = R*(0) [1+2 -i ] 

1 X R x (0) 

= R*(0) + 2R^(x) 



The autocovariance function of Y(t) is therefore 
C Y (x) = R y (t) - {E[Y(t k )]} 2 

= R x (0) + 2R x (x) - R x (0) 
= 2r2( t ) 



Problem 1.21 



C^) The random variable Y(t^) obtained by observing the filter output of impulse response 
h^t), at time is given by 



Y(t,) = / X(t r x) h^t) dx 



The expected value of Y(t.|) is 
By = E [Y(t^ ) ] 

:H,(0) 

where 



H^O) = / h.,(x) d T 
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r- 



The random variable Z(t 2 ) obtained by observing the filter output of impulse response 
h 2 (t), at time t 2 , is given by 



Z(t 2 ) = / X(t 2 -u) h 2 (u) du 



— 00 



The expected value of Z(t 2 ) is 

m 7 = E[z(t 9 )] 
2 



= H 2 (0) 

where 

00 

H 2 (0) = / h 2 (u) du 



The covariance of Y(t^) and Z(t 2 ) is 



Cov[Y(t 1 )Z(t 2 )] = ECCYCt^ -^ UZ(t 2 ) )] 



Elf I (X(t r T) -/i^) (X(t 2 -u) -/(y.) h^x) h 2 (u) dx du] 



/ / E[(X(t r x) -/t. x ) (X(t 2 -u) -Z^)] h^x) h 2 (u) dx du 



= / / C^t^-t 2 -T+u) h.,(x) h 2 (u) dx du 



—00 —00 



where C x (x) is the autocovariance function of X(t) . Next, we note that the variance of 



YCt^ is 



ol = E[(Y(t,) -/V ) 2 ] 
1 1 



= / / ^ x (x-^i) h^Cx) h (u) dx du 

—00 — 00 » 



and the variance of Z(t 2 ) is 



CT? = E[(Z(t.) -yu ) 2 ] 
L 2 d L 2 



- I I C x (x-u) h 2 (x) h 2 (u) dx du 

—00 —00 
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The correlation coefficient of Y(t.,) and Z(t 2 ) is 

covCYU^ZCt^)] 
p = 

Since X(t) is a Gaussian process, it follows that Y(t ) and Z(t ? ) are jointly Gaussian 
with a probability density function given by 

f Y(t 1 ),z(t 2 ) (y r z 2 ) = ^«pt-Q(y r z 2 )] 

where 



K 



2ira v a /1-p 
Y 1 z 2 



Q(y r z 2 ) = 



2 (1-p 2 ) 



y 1 7% l 2 



) - .2 pC 



Y 1^Y1 w z 2y^Z2, , Z 2~/ U 12,2 



-)( 



'2 J 



(b) The random variables Y(t^) and Z(t 2 ) are uncorrelated if and only if their covariance 
is zero. Since Y(t) and Z(t) are jointly Gaussian processes, it follows that Y(t.j) and 
Z(t 2 ) are statistically independent if Cov[Y(t 1 )Z(t 2 )] is zero. Therefore, the necessary 
and sufficient condition for Y(t^) and Z(t 2 ) to be statistically independent is that 



/ / C (t -tg-x-Hi) h^x) h 2 (u) dT du = 0 

—00 —00 



for choices of t 1 and t 2 . 
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Problem 1.22 

(a) The filter output is 

00 

Y(t) = / h( T ) X(t-x) dT 



1 T 

= f- / X(T-t) d T 
0 



Put T-x=u. Then, the sample value of Y(t) at t=T equals 
1 T 

Y = - / X(u) du 
0 

The mean of Y is therefore 
1 T 

E [Y ] = E [=- / X(u) du] 
0 



1 1 

7 / E[X(u)] du 
0 



The variance of Y is 

a 2 = E[Y 2 ] - {E[Y]} 2 
= R y (0) 



= / S y (f) df 



= / S x (f) IH(f) | 2 df 

— oo 
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But 



H(f) = / h(t) exp(-j2*ft) dt 

— 00 

1 1 

= y f exp(-j2nft) dt 
0 

1 r exp(-j2Tift) .. T 
" T -j2*f J Q 

= jlW C1 " exp(-j2rfT)] 
= sinc(fT) exp(-jTrfT) 

Therefore , 

OS 

o 2 = / S x (f) sinc 2 (fT) df 

-CO 

(b) Since the filter input is Gaussian, it follows that Y is also Gaussian. Hence, the 
probability density function of Y is 

1 y 2 
f Y (y) = ~ exp(- ^-g) 

/2ir <jy 2<jy 

2 

where 0 y is defined above. 



Problem 1 .7.3 

(a) The power spectral density of the noise at the filter output is given by 
s (f ) _ 5l 1 j2TrfL f 2 
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0 (2TrfL/R) 2 
: 1 + (2TrfL/R) 



[1 - 



1+(2TrfL/R) 



The autocorrelation function of the filter output is therefore 




(b) The mean of the filter output is equal to H(0) times the mean of the filter input. The process 
at the filter input has zero mean. The value H(0) of the filter's transfer function H(f) is zero. 
It follows therefore that the filter output also has a zero mean. 

The mean-square value of the filter output is equal to R N (0). With zero mean, it follows 
therefore that the variance of the filter output is 



= Rn<0) 



Since R^i) contains a delta function 6(t) centered on x = 0, we find that, in theory, o N 2 is 
infinitely large. 
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Problem 1.24 

(a) The noise equivalent bandwidth is 

oo 

W N = 1/2 / |H(f) | 2 df 
|H(0)! 2 -«» 



is df 



-» 1 + (f/fg) 



oo 

= / — ^ 



0 1 + (f/fQ) 2 " 



TTf 0 



2n sin(Tr/2n) 
f„ 



0 



sinc(1/2n) 

(b) When the filter order n approaches infinity, we have 



W N = f 0 lim sinc(1/2n) 



= f 0 



Problem 1.25 

The process X(f) defined by 



X(fc) = £ h(t - x k ), 

k=-oo 



where h(t - x k ) is a current pulse at time x k , is stationary for the following simple reason. There is 
no distinguishing origin of time. 
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Problem 1 .26 

(a) Let S^f) denote the power spectral density of the noise at the first filter output 
The dependence of S^f) on frequency is illustrated below: 



s 1 (f) 



V 2 




Let S 2 (f) denote the power spectral density of the noise at the mixer output. Then, we 
may write 

S 2 (f) = 1 CS 1 (f+f c ) + S^f-f )] 



which is illustrated below: 



s 2 (f) 



— i— 

■2f 



IB 



N Q /4 



2B 



V 8 



2f 



2B 
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The power spectral density of the noise n(t) at the second filter output is therefore denned by 



S 0 (f) = 



- 




-B<f<B 



[0, 



otherwise 



The autocorrelation function of the noise n(t) is 



R 0 (t) = 



N 0 B 



sinc(2Bx) 



2 



(b) The mean value of the noise at the system output is zero. Hence, the variance and mean-square 
value of this noise are the same. Now, the total area under S 0 (f) is equal to (Nq/4X2B) = N 0 B/2. The 
variance of the noise at the system output is therefore N 0 B/2. 

(c) The maximum rate at which n(t) can be sampled for the resulting samples to be uncorrelated is 
2B samples per second. 
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Problem 1.7.7 



(a) The autocorrelation function of the filter output is 

00 oo 

R x (x) = / / Mx.,) h(x 2 ) R w (t-t 1 +t 2 ) dx 1 dx 2 

CO _co 

Since RyCx) = (N Q /2) 6(x), we find that the impulse response h(t) of the filter must 
satisfy the condition: 

Nq oo oo 

R x (x) - 2" S S Mx.,) h(x 2 ) 5(t-t 1 +t 2 ) dx 1 dx 2 

—CO —CO 

N Q » 

= ~2~ S h(t+x 2 ) h(x 2 ) dx 2 

—CO 

(b) For the filter output to have a power spectral density equal to S x (f) , we have to 
choose the transfer function H(f) of the filter such that 

N 0 2 

s x (f) = Y~ »H(f>r 



or 



lH(f)l = 7-jr — 

0 
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Problem 1.28 

(a) Consider the part of the analyzer in Fig. 1.19 defining the in-phase component rij(t), 
reproduced here as Fig. 1: 



Narrowband 

noise 

n{t) 






Low-pass 






filter 


► 



2cos(27t/ c r) 



Figure 1 

For the multiplier output, we have 
v(f) = 2/i(f)cos(27i/ c f) 

Applying Eq. (1.55) in the textbook, we therefore get 
S v (f) = iS N (f-f c ) + S N (f + f c )] 

Passing v(t) through an ideal low-pass filter of bandwidth B, defined as one-half the bandwidth of 
the narrowband noise n(t), we obtain 



S N,(f) = 



S v (f) for -B<f<B 



0 



otherwise 



= S N (f-f c ) + S N (f + f c ) for -B<f<B 
{ 0 otherwise 

For the quadrature component, we have the system shown in Fig. 2: 



(1) 



Narrowband 

noise 

n(t) 




u(t) ^ 


Low-pass 






filter 


► 



n Q (t) 



-2sin(27t/ c /) 



Fig. 2 
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(2) 



The multiplier output u(t) is given by 
u{t) = -2n(t)sm(2nf c t) 
Hence, 

S v (f) = [S N (f-f c ) + S N (f + f c )] 
and 

S N (f) = \ S u(f) ^ -B<f<B 
Q I 0 otherwise 

= j S N (f-f c ) + S N (f + f c ) for -B < f <B 
I 0 otherwise 

Accordingly, from Eqs. (1) and (2) we have 



s N ,(f) = s Nq (D 



(b) Applying Eq. (1.78) of the textbook to Figs. 1 and 2, we obtain 

S Nl N Q (f) = \H(f)\ 2 S vu (f) (3) 
where 



\H(f)\ = 1 for -B-f-B 
{ 0 otherwise 

Applying Eq. (1.23) of the textbook to the problem at hand: 

i /27t/„T -]2nf x 

R VU (T) = 2R N (T)sin(2nf c T) = -.R N (x)(e -e c ) 

Applying the Fourier transform to both sides of this relation: 

SvuW - -j(S N (f-f c )-S N (f + f c )) (4) 
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Substituting Eq. (4) into (3): 

c m _ J jiS N (f + f c )-S N (f-f c )] for -B<f<B 
I 0 otherwise 

which is the desired result. 
Problem 1.29 

If the power spectral density S N (f) of narrowband noise n(f) is symmetric about the midband 
frequency f c we then have 

S N (f-f c ) = S N (f + f c ) for -B<f<B 

From part (b) of Problem 1.28, the cross-spectral densities between the in-phase noise component 
nj(t) and quadrature noise component ng(t) are zero for all frequencies: 

S N ,N Q (f) = 0 to all / 

This, in turn, means that the cross-correlation functions R^^^) and R N ^ N ^x) are both zero, 
that is, 

E[N I (t k + x)N Q (t k )] = 0 

which states that the random variables Nj(t k + x) and NQ(t k ), obtained by observing nj(t) at time 
t k + t and observing ng(r) at time t k , are orthogonal for all t. 

If the narrow-band noise n{t) is Gaussian, with zero mean (by virtue of the narrowband nature of 
n(0), then it follows that both Nj(t k + x) and A^g(^) are also Gaussian with zero mean. We thus 
conclude the following: 

• Nj{t k + x) and Nq(t k ) are both uncorrected 

• Being Gaussian and uncorrected, Njit k + x) and Nq{t k ) are therefore statistically independent. 

That is, the in-phase noise component nj(t) and quadrature noise component nqif) are statistically 
independent. 
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Problem 1.30 

(a) The power spectral density of the in-phase component or quadrature component is 
defined by 



S N (f+f c ) + S N (f-f c ) , -B < f < B 



S N (f) = S N (f) 



otherwise 



We note that, for -2 < f < 2, the S N (f+5) and S N (f-5) are as shown below: 



S N (f + 5) 




V f - 5) 



1.0 



-2 



We thus find that S N (f) or S N (f) is as shown below: 

S N (f)=S N (f) 




-2 -1 



(tO The cross-spectral density S N N (f) is defined by 

T Q 



j[S N (f+f c ) - S N (f-f c )] , -B < f < B 



s N N (f) = 
T S 



o , 



otherwise 



We therefore find that S N N (f)7j is as shown below: 



- S (f) 
3 N N 
X Q 




Next, we note that 



s NN (f) =s* N (f) 



I Q 



I Q 



We thus find that ^ N (f) is as shown below: 

J Q 





T S N N (f) 

J 6/ 


. 0.5 

1 


2 


-2 


-1 0 
-0.5 


\ 1 
\ 1 

\ 1 / 
\ ' / 





_ f 



Problem 1.31 

(a) Express the noise n(t) in terms of its in-phase and quadrature components as follows: 

n(t) = n (t) cos(2irf t) - n (t) sin(2nf t) 
X C Q c 

The envelope of n(t) is 



r(t) = /n 2 (t) + n 2 (t) 
which is Rayleigh-distributed. That is 

2 



f R (r) = 



r f T ~ v 

-r exp(- — ) , 
o 2a 



0, 



r > 0 



otherwise 



To evaluate the variance a , we note that the power spectral density of n (t) or n (t) is 
as follows J" $ 
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S N (f)=S N (f) 
I ft 



N. 



-B 



Since the mean of n(t) is zero, we find that 



o = 2 N Q B 



Therefore , 



f R (r) = 



r , r . 

2N Q B exp(l " 4NqB ' 

0 , 



r >_ 0 
othervd.se 



(b) The mean value of the envelope is equal to /itN n B, and its variance is equal to 
0.-858 N Q B. u 
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Problem 1.32 



Autocorrelation of a Sinusoidal Wave Plus White Gaussian Noise 

In this computer experiment, we study the statistical characterization of a random process X(t) 
consisting of a sinusoidal wave component Acos(2nf c t + ©) and a white Gaussian noise process 
W(t) of zero mean and power spectral density Nq/2. That is, we have 



where 0 is a uniformly distributed random variable over the interval(-7t,7i). Clearly, the two 
components of the process X(t) are independerit. The autocorrelation function of X(t) is therefore 
the sum of the individual autocorrelation functions of the signal (sinusoidal wave) component and 
the noise component, as shown by 



This equation shows that for |x| > 0, the autocorrelation function R x (t) has the same sinusoidal 
waveform as the signal component. We may generalize this result by stating that the presence of a 
periodic signal component corrupted by additive white noise can be detected by computing the 
autocorrelation function of the composite process X(t). 

The purpose of the experiment described here is to perform this computation using two different 
methods: (a) ensemble averaging, and (b) time averaging. The signal of interest consists of a 
sinusoidal signal of frequency f c = 0.002 and phase 0 = - n/2, truncated to a finite duration T = 
1000; the amplitude A of the sinusoidal signal is set to V2 to give unit average power. A particular 
realization x(t) of the random process X(t) consists of this sinusoidal signal and additive white 
Gaussian noise; the power spectral density of the noise for this realization is (Nq/2) = 1000. The 
original sinusoidal is barely recognizable in x(t). 

(a) For ensemble-average computation of the autocorrelation function, we may proceed as 



• Compute the product x(t + x)x(f) for some fixed time t and specified time shift T, where x{t) 
is a particular realization of the random process X(t). 

• Repeat the computation of the product x(t + x)x(t) for M independent realizations (i.e., 
sample functions) of the random process X(0- 

• Compute the average of these computations over M. 

• Repeat this sequence of computations for different values of x. 

The results of this computation are plotted in Fig. 1 for M - 50 realizations. The picture 
portrayed here is in perfect agreement with theory defined by Eq. (2). The important point to 
note here is that the ensemble-averaging process yields a clean estimate of the true 



X{t) = Acos2nf c t + e + W(t) 



(1) 




(2) 



follows: 
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autocorrelation function R x (t) of the random process X(t). Moreover, the presence of the 
sinusoidal signal is clearly visible in the plot of R x (t) versus t. 

(b) For the time-average estimation of the autocorrelation function of the process X(t), we invoke 
ergodicity and use the formula 



R x (t) = lim R x (x, T) (3) 

T — > °° 

where R x (x,T) is the time-averaged autocorrelation function: 

R x (x, T) = 5ff x(t + x)x{t)dt (4) 

The x(t) in Eq. (4) is a particular realization of the process X(t), and IT is the total observation 
interval. Define the time-windowed function 

x (,) = H 0 ' ~ T ^ t ^ T 
T [O, otherwise ^ 

We may then rewrite Eq. (4) as 

R X {%,T) = ^fj T {t + x)x T {t)dt (6) 

For a specified time shift x, we may compute R x (t,T) directly using Eq. (6). However, from a 
computational viewpoint, it is more efficient to use an indirect method based on Fourier 
transformation. First, we note From Eq. (6) that the time-averaged autocorrelation function 
R x (x,T) may be viewed as a scaled form of convolution in the x-domain as follows: 

R x (x, T) = ±Q* r (x)* x r (-x) (7) 

where the star denotes convolution and Xj(t) is simply the time- windowed function Xj{t) with 
t replaced by x. Let Xjij) denote the Fourier transform Xj{x)\ note that Xjif) is the same as the 
Fourier transform X(f,T). Since convolution in the x-domain is transformed into multiplication 
in the frequency domain, we have the Fourier-transform pair: 

R x (x,T) = ±jjx T (f)\ 2 (8) 
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The parameter \Xj(f)\ /2T is recognized as the periodogram of the process X(t). Equation (8) is 
a mathematical description of the correlation theorem, which may be formally stated as 
follows: The time-averaged autocorrelation function of a sample function pertaining to a 
random process and its periodogram, based on that sample function, constitute a Fourier- 
transform pair. 

We are now ready to describe the indirect method for computing the time-averaged 
autocorrelation function R x {x,T): 

• Compute the Fourier transform Xj(f) of time- windowed function x^x). 

• Compute the periodogram \Xj{f)\ 2 /2T. 

• Compute the inverse Fourier transform of \Xj(f)\ 2 /2T. 

To perform these calculations on a digital computer, the customary procedure is to use the fast 
Fourier transform (FFT) algorithm. With xj{x) uniformly sampled, the computational 
procedure described herein yields the desired values of R x (t,T) for 
x = 0, A, 2 A, (TV - 1)A where A is the sampling period and N is the total number of 
samples used in the computation. Figure 2 presents the results obtained in the time-averaging 
approach of "estimating" the autocorrelation function R x (?) using the indirect method for the 
same set of parameters as those used for the ensemble-averaged results of Fig. 1. The symbol 
Rx(t) is use d to emphasize the fact that the computation described here results in an 
"estimate" of the autocorrelation function R x (i)- The results presented in Fig. 2 are for a 
signal-to-noise ratio of + lOdB, which is defined by 

2 2 

SNR = A 72 = — (9) 
N 0 /(2T) N 0 K } 



On the basis of the results presented in Figures 1 and 2 we may make the following 
observations: 

• The ensemble-averaging and time-averaging approaches yield similar results for the 
autocorrelation function i?^(x), signifying the fact that the random process X(t) described 
herein is indeed ergodic. 

• The indirect time-averaging approach, based on the FFT algorithm, provides an efficient 
method for the estimation of R x (t) using a digital computer. 

• As the SNR is increased, the numerical accuracy of the estimation is improved, which is 
intuitively satisfying. 
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1 Problem 1.32 



Matlab codes 

*/. Problem 1.32a CS : Haykin 

'/. Ensemble average autocorrelation 

*/, M. Sellathurai 



clear all 
A=sqrt(2); 

N=1000; M=l; SNRdb=0; 
e_corrf_f=zeros(l, 1000) ; 
f_c=2/N; 
t=0:l:N-l; 

for trial=l:M 

*/, signal 

s=cos(2*pi*f_c*t) ; 
'/.noise 

snr = 10~(SNRdb/10); 

wn = (randn(l,length(s)))/sqrt(snr)/sqrt(2) ; 

'/.signal plus noise 
s=s+wn; 

'/, autocorrelation 
[e_corrf]=en_corr(s,s, N) ; 

'/.Ensemble-averaged autocorrelation 
e_corrf _f =e_corrf _f +e_corrf ; 
end 

'/.prints 

plot (-500: 500-1, e_corrf_f/M) ; 
xlabel('(\tau)') 
ylabel('R_X(\tau) ') 
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'/. Problem i.32b CS: Haykin 

'/, time-averaged estimation of autocorrelation 
'/. M. Sellathurai 



clear all 
A=sqrt(2) ; 
N=1000; SNRdb=0; 
f_c=2/N; 
t=0: 1:N-1; 

'/. signal 

s=cos(2*pi*f_c*t) ;'/,noise 
'/.noise 

snr = 10~(SNRdb/10) ; 

wn = (randn(l,length(s)))/sqrt(snr)/sqrt(2); 

'/.signal plus noise 
s=s+wn; 

'/, time -averaged autocorrelation 
[e_corrf]=time_corr(s,N) ; 

'/.prints 

plot (-500 : 500-1 , e_corrf ) ; 
xlabel('(\tau)>) 
ylabel('R_X(\tau) ') 



45 



function [corrf ]=en_corr(u, v, N)'/. funtion to compute the autocorreation/ cross-correlatic 

'/, ensemble average 

*/. used in problem 1.32, CS: Haykin 

'/. M. Sellathurai, 10 june 1999. 

max_cross_corr=0 ; 
tt=length(u) ; 

for m=0:tt 

shifted_u=[u(m+l:tt) u(l:m)]; 

corr(m+l)=(sum(v.*shifted_u))/(N/2) ; 

if (abs(corr)>max_cross_corr) 

max_cross_corr=abs (corr) ; 

end 

end 

corrl=f lipud(corr) ; 
corrf=[corrl(501:tt) corr(l:500)] ; 
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function [corrf]=time_corr(s ,N) 

'/, funtion to compute the autocorreation/ cross-correlation 
7, time average 

'/. used in problem 1.32, CS: Haykin 
*/. M. Sellathurai, 10 june 1999. 

X=fft(s) ; 

Xi=fftshift((abs(X) . ~2)/(N/2)) ; 
corrf =(f f tshif t (abs ( if ft (XI ) ) ) ) ; 
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Answer to Problem 1.32 



1.5- 




Figure 1: Ensemble averaging 




Figure 2: Time averaging 
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Problem 1.33 



Matlab codes 

'/. Problem i.33 CS: Haykin 
'/, multipath channel 
*/. M. Sellathurai 

clear all 

Nf=0;Xf=0; '/, initializing counters 

N=10000; '/, number of samples 
M=2; P=10; 

a=l; '/, line of sight component component 
for i=l:P 

A=sqrt(randn(N,M) . ~2 + randn(N.M) . "2) ; 

xi=A.*cos(cos(rand(N,M)*2*pi) + rand(N,M)*2*pi) ; '/. inphase cpmponent 
xq=A.*sin(cos(rand(N,M)*2*pi) + rand(N,M)*2*pi) ; '/. quadrature phase component 

xi=(sum(xi ' ) ) ; 
xq=(sum(xq' )) ; 

ra=sqrt((xi+a) . '2+ xq.*2) ; */, rayleigh, rician fading 

[h X]=hist(ra,50); 

Nf=Nf+h; 
Xf=Xf+X; 

end 

Nf=Nf/(P); 
Xf=Xf/(P); 

'/, print 

plot (Xf , Nf / (sum(Xf . *Nf ) /20) ) 
xlabel('v') 
ylabel('f_v(v) ') 
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Answer to Problem 1.33 




V 



Figure | Rician distribution 
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CHAPTER 2 



Continuous-wave Modulation 



Problem 2.1 



(a) Let the input voltage v i consist of a sinusoidal wave of frequency ^ f (i.e., half 
the desired carrier frequency) and the message signal m(t): 



v. = cos(irf t)+m(t) 
1 c c 

Then, the output current i Q is 
i 0 = a, v. + a 3 vj 5 



= a. [A cosdrf t)+m( t) ]+a„[A cosGr f t)+m(t)] 3 

= a 1 [A c cos(Trf c t)+m(t)] + ^A 3 [cos(3ir f c t)+3cos(Tr f,t) ] 

3 2 ? 
+ 2 a -^ c m(t)[1+cos(2irf c t)] + 3a 3 A c cos(7rf c t)m (t) + a^U) 

Assume that m(t) occupies the frequency interval -W _< f _< W. Then, the amplitude spectrum 



of the output current i is as shown below. 

o 



I (f) 

o 



f A A. A 




k A i 


3f 

c 

2 


- f 

c 




f 

c 
2 


-3W -WOW 3W 


f 

c 
2 




f 

c 


3f 

c 

2 



2W 



4W 



4W 



2W 
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From this diagram we see that in order to extract a DSBSC wave, with carrier frequency f 

c 

from i Q , we need a band-pass filter with mid-band frequency f and bandwidth 2W, which 
satisfy the requirement: 



f - W > + 2W 
that is, f > 6W 

Therefore, to use the given nonlinear device as a product mmodulator, we may use the 
following configuration: 



Nonlinear 
device 



BPF 



A COS (TTf fc)i 

c c 



m(t)' 



3 2 

— a A m(t) cos(2irf t) 
^ j c c 



(b) To generate an AM wave with carrier frequency f c we require a sinusoidal component of 
frequency f Q to be added to the DSBSC generated in the manner described above. To achieve 
this requirement, we may use the following configuration involving a pair of the nonlinear 
devices and a pair of identical band-pass filters. 



A COS (TTf t) 
C C 



m(t) 



A cos(irf t) 
c c 



Nonlinear 
device 



BPF 



Nonlinear 
device 



BPF 



A 



AM wave 
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The resulting AM wave is therefore f A c [A 0 +m(t) ]cos(2irf c t) . Thus, the choice of the dc 
level A Q at the input of the lower branch controls the percentage modulation of the AM 
wave. 



Problem 2.2 

Consider the square-law characteristic: 



v 2 (t) = a^Ct) + a 2 v 1 (t) 



(1) 



where a x and a 2 are constants. Let 

vj(t) = A c cos(27tf c t) + m(t) 
Therefore substitutingEq. (2) into (1), and expanding terms: 



(2) 



v 2 (t) = aiA ( 



2a 2 , x 
1 + — £ m(t) 



COS(27lf c t) 



(3) 



+ ajm(t) + a 2 m 2 (t) + a 2 A c cos 2 (2rcf c t) 



The first term in Eq. (3) is the desired AM signal with k a = 2a^a v The remaining three terms are 
unwanted terms that are removed by filtering. 

Let the modulating wave m(t) be limited to the band -W < f < W, as in Fig. 1(a). Then, from Eq. (3) 
we find that the amplitude spectrum I V 2 (f) I is as shown in Fig. 1(b). It follows therefore that the 
unwanted terms may be removed from v 2 (t) by designing the tuned filter at the modulator output 
of Fig. PS. 2 to have a mid-band frequency f c and bandwidth 2W, which satisfy the requirement that 
f c >3W. 



\M(f)\ 



\V 2 (f)\ 



-W 0 W f 

(a) 
Figure 1 



-2f c -f c -2W WOW 2W 

(b) 



fc 

2W 



2fc fc 
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Problem 2.3 



The generation of an AM wave may be accomplished using various devices; here we describe one 
such device called a switching modulator. Details of this modulator are shown in Fig. P2.3a, 
where it is assumed that the carrier wave c{t) applied to the diode is large in amplitude, so that it 
swings right across the characteristic curve of the diode. We assume that the diode acts as an ideal 
switch, that is, it presents zero impedance when it is forward-biased [corresponding to c(t) > 0]. 
We may thus approximate the transfer characteristic of the diode-load resistor combination by a 
piecewise-linear characteristic, as shown in Fig. P2.3b. Accordingly, for an input voltage v^f) 
consisting of the sum of the carrier and the message signal: 

v t (0 = A c cos(27i/ c f) + /n(0 (1) 

where \m{i)\ « A c , the resulting load voltage v 2 (t) is 

V2(r) U(0, c(o>o (2) 

I 0, c(t)<0 

That is, the load voltage v 2 (t) varies periodically between the values vjO) and zero at a rate equal 
to the carrier frequency f c . In this way, by assuming a modulating wave that is weak compared 
with the carrier wave, we have effectively replaced the nonlinear behavior of the diode by an 
approximately equivalent piecewise-linear time-varying operation. 

We may express Eq. (2) mathematically as 

v 2 (f)= A c cos(2nf c t) + m(t)g To (t) (3) 

where gj- Q (t) is a periodic pulse train of duty cycle equal to one-half, and period T Q - \lf c , as in 
Fig. 1. Representing this g To (t) by its Fourier series, we have 

°° n-l 

*r 0 (0 = 5 + |X^^cos[27t/ c f(2ii-l)] (4) 

n=l 

Therefore, substituting Eq. (4) in (3), we find that the load voltage v 2 (t) consists of the sum of two 
components: 



1 . The component 

z(0j cos(27t/ c r) 



1 4 / 

1 — m( 
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which is the desired AM wave with amplitude sensitivity k a = 4nA c . The switching modulator 
is therefore made more sensitive by reducing the carrier amplitude A c ; however, it must be 
maintained large enough to make the diode act like an ideal switch. 

2. Unwanted components, the spectrum of which contains delta functions at 0, ±2f c , ±4f c , and so 
on, and which occupy frequency intervals of width 2W centered at 0, ±3f c . ±5f c , and so on, 
where W is the message bandwidth. 





gT 0 C> 
+ 1 


— 1 1 1 .J 






0 la T 0 t 



Fig. 1: Periodic pulse train 

The unwanted terms are removed from the load voltage v 2 (0 by means of a band-pass filter with 
mid-band frequency f c and bandwidth 2W, provided that/ c > 2W. This latter condition ensures that 
the frequency separations between the desired AM wave the unwanted components are large 
enough for the band-pass filter to suppress the unwanted components. 
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Problem 2.4 

(a) The envelope detector output is 

v(t) = A |1+ ycos(2rrf t) | 
c tn 

which is illustrated below for the case when p=2. 




We see that v(t) is periodic with a period equal to f , and an even function of t, and so 
we may express v(t) in the form: m 



v(t) = a n + 2 E a„ cos(2nirft) 
u „ n m 

n=1 



1/2f 

where a. = 2f / m v(t)dt 
0 m Q 



1 /3f _ 1 /2f 

2A f / [1+2 cos(2irf t)]dt + 2A f / m [-1 -2cos(2irf t)]dt 

c m q - m c m ^ ^ m 

m 



A 4A _ 

c c . , df{ . 
-o + sin(-%) 



(1) 



1/2f 

a„ = 2f / v(t)cos(2mrf t)dt 

n m g m 



1/3f 

2A c f m / m [1+2cos(2Trf m t)]cos(2mrf m t)dt 
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1/2f 

+ 2A f / m [-1-2cos(2irf t)]cos(2nirf t)dt 

1 /3f m 
m 

A A 
= "n7 [2 sin (^ - sin(m)3 + j^pfr {2 sin[^-(n+1)] - sinCir (n+1 )]} 

+ (n-°)ir {2 sin t^(n-1)3 - sinCir (n-1 )] } (2) 
For n=0, Eq. (2) reduces to that shown in Eq. (1). 
(b) For n=1, Eq. (2) yields 

1 c 2tt 3 
For n=2, it yields 



A/3 
a 2 = — 

Therefore, the ratio of second-harmonic amplitude to fundamental amplitude in v(t) is 

t=\ = 3/3 = 0.452 
3 1 2„ + 3/3 
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Problem 2.5 



(a) The demodulation of an AM wave can be accomplished using various devices; here, we 
describe a simple and yet highly effective device known as the envelope detector. Some 
version of this demodulator is used in almost all commercial AM radio receivers. For it to 
function properly, however, the AM wave has to be narrow-band, which requires that the 
carrier frequency be large compared to the message bandwidth. Moreover, the percentage 
modulation must be less than 100 percent. 

An envelope detector of the series type is shown in Fig. P2.5, which consists of a diode and a 
resistor-capacitor (RC) filter. The operation of this envelope detector is as follows. On a 
positive half-cycle of the input signal, the diode is forward-biased and the capacitor C charges 
up rapidly to the peak value of the input signal. When the input signal falls below this value, 
the diode becomes reverse-biased and the capacitor C discharges slowly through the load 
resistor ify. The discharging process continues until the next positive half-cycle. When the 
input signal becomes greater than the voltage across the capacitor, the diode conducts again 
and the process is repeated. We assume that the diode is ideal, presenting resistance /y to 
current flow in the forward-biased region and infinite resistance in the reverse-biased region. 
We further assume that the AM wave applied to the envelope detector is supplied by a voltage 
source of internal impedance R s . The charging time constant (ry + R s ) C must be short 
compared with the carrier period l// c , that is 



(r f + R s )C« T (1) 

J c 

so that the capacitor C charges rapidly and thereby follows the applied voltage up to the 
positive peak when the diode is conducting. 

(b) The discharging time constant R[C must be long enough to ensure that the capacitor 
discharges slowly through the load resistor /?; between positive peaks of the carrier wave, but 
not so long that the capacitor voltage will not discharge at the maximum rate of change of the 
modulating wave, that is 

where W is the message bandwidth. The result is that the capacitor voltage or detector output 
is nearly the same as the envelope of the AM wave. 
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Problem 2.6 



Let 

vjft) = A c [l + k a m(t)]cos(27tf c t) 



(a) Then the output of the square-law device is 

v 2 (t) = a^t) + a 2 v 1 2 (t) 

= a^^l + k a m(t)]cos(2rcf c t) 

+ la 2 A c 2 [l + 2k a m(t) + k 2 m 2 (t)] [1 + cos(4jrf c t)] 



(b) The desired signal, namely agA,. k a m(t), is due to the agVj (t) - hence, the name "square-law 
detection". This component can be extracted by means of a low-pass filter. This is not the only 
contribution within the baseband spectrum, because the term 1/2 a2A c 2 k a 2 m 2 (t) will give rise to a 
plurality of similar frequency components. The ratio of wanted signal to distortion is 2/k a m(t). To 
make this ratio large, the percentage modulation, that is, I k a m(t) T should be kept small compared 
with unity. 
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Problem 2.1 



The squarer output is 

Vl (t) = A 2 [1+k a m(t)] 2 cos 2 (2irf t) 



= 5- [1+2k m(t)+m (t)][1+cos(%f t)] 



The amplitude spectrum of v.. (t) is therefore as follows, assuming that m(t) is limited to 
the interval -W < f < W: 




|v i (f) 




4W 




Since f > 2W, we find that 2f -2W > 2W. Therefore, by choosing the cutoff frequency of 
c c 

the low-pass filter greater than 2W, but less than 2f -2W, we obtain the output 



v 2 (t) = y- [1-tk a m(t)]' 



Hence, the square-rooter output is 



v,(t) = — m(t)] 
3 a 



which, except for the dc component — , is proportional to the message signal m(t). 
Problem 2.8 



(a) For f = 1.25 kHz, the spectra of the ( message signal m(t), the product modulator 



output s(t), and the coherent detector output v(t) are as follows, respectively: 
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M(f) 





f (kHz) 



-f (kHz) 



V(f) 



To avoid sideband-overlap, the carrier frequency t Q must be greater than or equal 
1 kHz. The lowest carrier frequency is therefore 1 kHz for each sideband of 
modulated wave s(t) to be uniquely determined by m(t). 

Problem 2. Q 

The two AM modulator outputs are 



Si(t) 



A c [l + k a m(t)]cos(27cf c t) 



s 2 (t) 



AJ1 - k a m(t)]cos(27if c t) 



Subtracting SgCt) from s 1 (t): 



8(t) = S 2 (t) - S](t) 



= 2k a A c m(t)cos(27if c t) 



which represents a DSB-SC modulated wave. 



62 



Problem 2.10 

(a) Multiplying the signal by the local oscillator gives: 

s-Ct) = A m(t) cos(2irf t) cos[2ir(f +Af)t] 
1 c c 

A 

= J— m(t) {cos(2TrAft) + cos[2ir(2f c +Af)t] } 

Low pass filtering leaves: 
A 

s 2 (t) = — m(t) cos(2irAft) 



Thus the output signal is the message signal modulated by a sinusoid of frequency 



Af. 



(b) If m(t) = cos(2irf t) , 



m 



then s 2 (t) = y~ cos(2uf m t) cos(2irAft) 



Problem 2.11 



(a) y(t) = s*(t) 




A 2 cos 2 (2 7r f t)m 2 (t) 
c C 



[1+cos(4nf t)]m 2 (t) 
<- c 



Therefore, the spectrum of the multiplier output is 



A 2 » 
c 



Y(f) = ^ / M(A)M(f-x)dX + Tp [/ M(x)M(f-2f -A)dX + / M( X)M(f+2f -X)dX] 



where M(f) = F[m(t)]. 
(b) At f=2f , we have 
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2 

A «> 

Y(2f c )=2^/ M( A)M(2f c -X)dA 

_ 00 

00 00 

+ -ip [/ M(X)M(-X)dX + / M(X)M(4f -X)dX] 

-00 _oo C 

Since M(-X) r M*(X), we may write 
A 2 » 

Y(2f ) = —- I M(X)M(2f -X)dX 

c 2 _00 c 

A 2 00 00 

+ — [/ |M(x)rdx+ / M( x)M(4f c -x)dX] (1 ) 

— 00 — 00 

With m(t) limited to -W < f _< W and f > W, we find that the first and third integrals 
reduce to zero, and so we may simplify Eq . (1) as follows 

°° 

Y(2f ) = ^ / |M(X)( 2 dX 



—00 



A 2 E 
c 



where E is the signal energy (by Rayleigh's energy theorem). Similarly, we find that 



A 2 

Y(-2f ) = tt E 

C H 



The band-pass filter output, in the frequency domain, is therefore defined by 



A 2 

V(f) = -jp E Af[6(f-2f c ) + 6(f+2f c )] 



Hence , 

2 



A' 

v(t) = —■ E Af cos(4rf t) 
2 c 



P roblem 2.12 

The multiplexed signal is 

s(t) = A c m 1 (t) cos(2itf c t) + A Q m 2 (t) sin(2itf t) 

Therefore, 

A A 
S(f) [M^f-f^+M^f+f,)] [M 2 (f-f c )-M 2 (f+f o )] 

where M^f) = F[m.,(t)] and Mp(f) = FCm-Ct)]. The spectrum of the received signal is 
therefore 

R(f) = H(f)S(f) 

A c 1 1 

= T H(f) [M^f-f^M^f+f,)* 1 M 2 (f-f c )- 1 M 2 (f + f c )] 

To recover m., (t) , we multiply r(t), the inverse Fourier transform of R(f) , by cos(2irf t) 
and then pass the resulting output through a low-pass filter, producing a signal with 8he 
following spectrum 

F[r(t)cos(2nf t)] = I [R(f-f )+R(f+f )] 

C <£ C C 

A 

= -jp HCf-f.KM^f-^) + M^f) + 1 M 2 (f-2f c ) - j M 2 (f)] 

A c 1 1 

+ — H(f+f c )[M 1 (f) +M 1 (f+2f c ) + jM 2 (f) - jH 2 (f+2f c )] (1) 

The condition H(f +f) = H«(f -f) is equivalent to H(f+f )=H (f-f ) ; this follows from the 
*» c c 

fact that for a real-valued impulse response h(t), we have H(-f)=H»(f). Hence, 
substituting this condition in Eq. (1), we get 

A 

F[r(t)cos(2Tif t)] = ^HCf-f )M,(f) 
c d c 1 



+ ^ H{t-t o )lH^f-2f Q ) + j M 2 (f-2f c )+M 1 (f+2f c ) - 1 M 2 (f+2f c >] 

The low-pass filter output, therefore, has a spectrum equal to U c /2) H(f-f )M.j(f). 

Similarly, to recover rn^t), we multiply r(t) by sin(2irf c t), and then pass the 

resulting signal through a low-pass filter. In this case, we get an output with a 

spectrum equal to (A /2) H(f-f )M 0 (f). 

c c 2 
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Problem 2.13 

When the local carriers have a phase error ^, we may write 
cos(2Trf c t +<}>) = cos(2Trf c t)cos <j, - sin(2irf c t) sin 4, 
In this case, we find that by multiplying the received signal r(t) by cos(2nf c t+ <j>) , 

and passing the resulting output through a low-pass filter, the corresponding low-pass 

filter output in the receiver has a spectrum equal to (A /2) H(f-f ) [cos <j> M.. ( f ) - sin <b 

c c 1 * 

M 2 (f)]. This indicates that there is cross-talk at the demodulator outputs. 
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Problem 2.14 

The transmitted signal is given by 

s(t) = A c m { (t) cos {2nf c t) + A c m 2 (t) sin (2nf c t) 

= A C [V 0 + m^t) + m r (t)]cos(2n f c t) + A^m^t) - m r (t)] sm(2nf c t) 
(a) The envelope detection of s(t) yields 

= AjiVQ + m^ + m^f + imjit) - m r (t)f 




To minimize the distortion in the envelope detector output due to the quadrature component, we 
choose the DC offset V 0 to be large. We may then approximate y^t) as 

which, except for the DC component A c V 0 , is proportional to the sum m^) + m r (t). 

(b) For coherent detection at the receiver, we need a replica of the carrier A c cos(27T/ c r). This 
requirement can be satisfied by passing the received signal s(f) through a narrow-band filter of 
mid-band frequency f c . However, to extract the difference m t {t) - m r (i), we need sin(27t/ c r), which 
is obtained by passing the narrow-band filter output through a 90°-phase shifter. Then, multiplying 
s(t) by sin(27i/ c r) andt low-pass filtering, we obtain a signal proportional to m t (t) - m r {t). 

(c) To recover the original loudspeaker signals m/(7) and m r (t), we proceed as follows: 

• Equalize the outputs of the envelope detector and coherent detector. 

• Pass the equalized outputs through an audio demixer to produce m z (0 and m r (t). 
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Problem 2.15 



(a) s(t) = AJl +k a m(t)) cos (2n ft) 



A. 



k A 

I i + tV 



COS (271/ t) 



To ensure 50 percent modulation, k a - 1, in which case we get 



s{t) = A c [ l + )cos(27t/ c O 
1 + 1 



(b) s(t) = A m(t)cos(2nf t) 



A, 



1 +t 



-cos(2n ft) 



A c 

(c) s(t) = -j[m(t)cos(2nf c t)-m(t)sm(2nf c t)] 



A, 



- cos (2tc/ t) rsin(27t/ c 0 



2 v J c" ' 9 L 

l+t l+t 



(d) s(t) = y c 



— i-xcos(27r/ c 0 + — ^sin(27t/ c 0 
■l+t l + t 



As an aid to the sketching of the modulated signals in (c) and (d), the envelope of either SSB 
wave is 



a(t) = 



1 t + 1 11 



Plots of the modulated signals in (a) to (d) are presented in Fig. 1 on the next page. 
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Problem 2.16 

Consider first the modulated signal 

s(t) = ^m(t)cos(2nf c t) - ^m(t)sin(2nf c t) 



(1) 



Let S(f) = F[s(t)], M(f) = F[m(t)], and M(f) = f[m(t)] where m(t) is the Hilbert transform of 
the message signal m(t). Then applying the Fourier transform to Eq. (1), we obtain 

S(f) = \lM(f-f c ) + M(f + f c )]-j-.[M(f-f c )-M(f + f c )] (2) 

From the definition of the Hilbert transform, we have 
M(f) = -jsga(f)M(f) 

where sgn(/) is the signum function. Equivalently, we may write 
-j&(f-f c ) = sgn(/-/ c )M(/-/ c ) 

-jM(f + f c ) = sgn(f + f c )M(f + f c ) 

(i) From the definition of the signum function, we note the following for/> 0 = and/>/ c : 
sgn(/-/ c ) = sgn(/ + / c ) = +1 

Correspondingly, Eq. (2) reduces to 

S(f) = \[M(f-f c )+M(f + f c )] + ±[M(f-f c )-M(f + f c )] 

In words, we may thus state that, except for a scaling factor, the spectrum of the modulated 
signal s(t) defined in Eq. (1) is the same as that of the DSB-SC modulated signal for f>f c . 

(ii) For f> 0 and/</ c , we have 
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sgn(/-/ c ) = -1 
sgn(/ + / c ) = +1 



Correspondingly, Eq. (2) reduces to 

S(f) = \lM(f-f c ) + M(f + f c )] + l -[-M{f-f c )-M(f-f c )} 
= 0 

In words, we may now state that for f< f c , the modulated signal s(t) defined in Eq. (1) is zero. 

Combining the results for parts (i) and (ii), the modulated signal s(t) of Eq. (1) represents a single 
sideband modulated signal containing only the upper sideband. This result was derived for/> 0. 
This result also holds for /< 0, the proof for which is left as an exercise for the reader. 

Following a procedure similar to that described above, we may show that the modulated signal 
s(t) = ^m(t)cos(2nf c t) + ^m(t)sm(2nf c t) (3) 
represents a single sideband modulated signal containing only the lower sideband. 
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Problem 2.17 



An error Af in the frequency of the local oscillator in the demodulation of an SSB signal, measured 
with respect to the carrier frequency f c , gives rise to distortion in the demodulated signal. Let the 
local oscillator output be denoted by A(. cos(27c(f c + Af)t). The resulting demodulated signal is given 
by (for the case when the upper sideband only is transmitted) 

v 0 (t) = 1 AgAg [m(t)cos(2jtAft) + m(t)sin(27iAft)] 

This demodulated signal represents an SSB wave corresponding to a carrier frequency Af. 
The effect of frequency error Af in the local oscillator may be interpreted as follows: 

X a ) If the SSB wave s(t) contains the upper sideband and the frequency error Af is positive, or 
equivalently if s(t) contains the lower sideband and Af is negative, then the frequency 
components of the demodulated signal v 0 (t) are shifted inward by the amount Af compared 
with the baseband signal m(t), as illustrated in Fig. lQ>). 

(b) If the incoming SSB wave s(t) contains the lower sideband and the frequency error Af is 
positive, or equivalently if s(t) contains the upper sideband and Af is negative, then the 
frequency components of the demodulated signal v 0 (t) are shifted outward by the amount Af, 
compared with the baseband signal m(t). This is illustrated in Fig. lc for the case of a 
baseband signal (e.g., voice signal) with an energy gap occupying the interval -f a < f < f a , in 
part (a) of the figure. 
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Problem 2.18 

(a,b) The spectrum of the message signal is illustrated below: 



MC*) 




Correspondingly, the output of the upper first product modulator has the following 
spectrum: 




The output of the upper low pass filter has the spectrum: 
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The output of the lower low pass filter has the spectrum: 



-4+4 




t 


2J + f i 


0 


0 U 



The output of the upper second product modulator has the spectrum: 




The output of the lower second product modulator, has the spectrum: 




Adding the two second product modulator outputs, their upper sidebands add constructively 
while their lower sidebands cancel each other. 

(c) To modify the modulator to transmit only the lower sideband, a single sign change is 
required in one of the channels. For example, the lower first product modulator could 
multiply the message signal by -sin(2nf t). Then, the upper sideband would be cancelled 
and the lower one transmitted. 
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Problem 2.19 



m(t) 



Product 
modulator 



K r <t) 



High-pass 
filter 



|v 2 (t) 



COS(2lTf t) 

c 



Product 
modulator 



f^(t) 



Low-pass 




filter 





s(t) 



cos [2ir(f +f )t] 
c b 



(a) The first product modulator output is 

v v (t) = m(t) cos(2trf t) 
The second product modulator output is 

v 3 (t) = v 2 (t) cos[2Tr(f c +f b )t] 

The amplitude spectra of m(t), v 1 (t) , v 2 (t), v^Ct) and s(t) are illustrated on the next 
page : 

We may express the voice signal m(t) as 
m(t) = J- [m + (t) + mjt)] 

« 

where m + (t) is the pre-envelope of m(t), and m_(t) ? m + (t) is its complex conjugate. The 

Fourier transforms of m + (t) and m (t) are defined by (see Appendix 2) 



M + (f) 



M (f) 




f > 0 

f < 0 

f > 0 

f < 0 



Comparing the spectrum of s(t) with that of m(t), we see that s(t) may be expressed in 
terms of m + (t) and m_(t) as follows: 

s(t) = g- m + (t)exp(-j2Tif b t)+ J- m_(t)exp( j2uf fa t) 

= [m(t)+jm(t)]exp(-j2Tif b t)+ ^{m( t)-jni( t)]exp( j27rf b t) 

= | m(t)cos(2nf b t)+ j Ht) sin(2*f b t) 



(b) With s(t) as input, the first product modulator output is 
v.,(t) = s(t) cos(2Tif c t) 
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|M(f) | 




-f -f °f f 
b a a b 



|v 1 (f)| 




±|M(f a ) 



-f 



0 f -f. 



/ 



c b / c \ c b 



f \ f +f. 



|v 2 (f)| 


• r -r 
c a 




f|M(f a )| 




i 

-l 


"T 









•f -f f 

c b c 



c c b 



|v 3 (f) 




■7 M(f ) 
4 ' a 




-f+f 0 f -f 
lo a b a 



S(f) 



- M(f ) 
4 1 a 




-f+f 0 f.-f 

b a b a 



2f +2f 
c b 
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Problem 2.20 

(a) Consider the system described in Fig. la, where u(t) denotes the product modulator output, as 
shown by 

u(t) = A c m(t) cos (2nf c t) 



Message 
signal 
m(t) 



Si- 


Product 


u(t) 


Band-pass 
filter 
H(f) 






modulator 


>• 


5*- 















A c cos (2nf c t) 



(a) 



Modulated 
signal 
s(t) 



signal 
s(t) 



d 

>■ 


Product 


v(t) 


Low-pass 






modulator 


=»- 


filter 


















A^cos (2nf c t) 



(b) 



Demodulated 
signal 



Figure 1: (a) Filtering scheme for processing sidebands, (b) Coherent detector for 
recovering the message signal. 

Let H(f) denote the transfer function of the filter following the product modulator. The spectrum 
of the modulated signal s(t) produced by passing u{t) through the filter is given by 

S(f) = U(f)H(f) 



= y[M(/-/ c ) + M(/ + / c )]//(/) 



(1) 



where M(f) is the Fourier transform of the message signal m{t). The problem we wish to address is 
to determine the particular H(f) required to produce a modulated signal s(f) with desired spectral 
characteristics, such that the original message signal m(t) may be recovered from s(t) by coherent 
detection. 



The first step in the coherent detection process involves multiplying the modulated signal s(t) by a 
locally generated sinusoidal wave A' c cos(27t/ c f), which is synchronous with the carrier wave 
A c cos(27t/' c 0, in both frequency and phase as in Fig. lb. We may thus write 
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v(t) = A / c cos(27t/ c O^(0 

Transforming this relation into the frequency domain gives the Fourier transform of v(t) as 

V(f) = ^lS(f-f c ) + S(f + f c )] (2) 

Therefore, substitution of Eq. (1) in (2) yields 
A A' 

V(f) = -^M(f)[H(f-f c ) + H(f + f c )] 



A A' 

+ -^[M(f - 2f c )H(f - f c ) + M(f + 2f c )H(f + f c )] (3) 

(b) The high-frequency components of v(t) represented by the second term in Eq. (3) are removed 
by the low-pass filter in Fig. lb to produce an output v 0 (f), the spectrum of which is given by the 
remaining components: 

A C A' 

V 0 {f) = -y 5 l M(f)[H(f-f c )+H(f + f c )] (4) 

For a distortionless reproduction of the original baseband signal m(t) at the coherent detector 
output, we require V 0 (f) to be a scaled version of M(f). This means, therefore, that the transfer 
function H(f) must satisfy the condition 

H(f-f c ) + H(f + f c ) = 2H(f c ) (5) 

where H(f c ), the value of H(f) at /= f c , is a constant. When the message (baseband) spectrum M(f) 
is zero outside the frequency range -W<f< W, we need only satisfy Eq. (5) for values of fin this 
interval. Also, to simplify the exposition, we set H(f c ) = 1/2. We thus require that H(f) satisfies the 
condition: 

H(f-f c ) + H(f + f c )=l, -W<f<W (6) 

Under the condition described in Eq. (6), we find from Eq. (4) that the coherent detector output in 
Fig. lb is given by 

A A' 

v o (0 = ( 7 ) 
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Equation (1) defines the spectrum of the modulated signal s(t). Recognizing that s(f) is a band- 
pass signal, we may formulate its time-domain description in terms of in-phase and quadrature 
components. In particular, s(t) may be expressed in the canonical form 

s(t) = ^(Ocos(27i/ c O-^ e (Osin(27t/ c O (8) 

where s 7 (r) is the in-phase component of s(f), and sg(t) is its quadrature component. To determine 
sj(t), we note that its Fourier transform is related to the Fourier transform of s(t) as follows: 

[S(f-f c ) + S(f + f\ -W<f<W 
Sj(f) = c (9) 

[ 0, elsewhere 

Hence, substituting Eq. (1) in (9), we find that the Fourier transform of sfa) is given by 
S,(f) = l -A c M(f)[H(f-f c ) + H(f + f c )] 

= l -A c M{f); -W<f<W (10) 

where, in the second line, we have made use of the condition in Eq. (6) imposed on H(f). From Eq. 
(10) we readily see that the in-phase component of the modulated signal s(t) is defined by 

s,(t) = ^A c m(t) (11) 

which, except for a scaling factor, is the same as the original message signal m(t). 

To determine the quadrature component s Q (t) of the modulated signal s(t), we recognize that its 
Fourier transform is defined in terms of the Fourier transform of s(t) as follows: 

[j[S(f-f c )-S(f + f c )] -W<f<W 
S Q (f) = c (12) 

[ 0, elsewhere 

Therefore, substituting Eq. (11) in (12), we get 

S Q (f) = U c M(f)[H(f-f c )-H(f + f c )] (13) 
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This equation suggests that we may generate sgit), except for a scaling factor, by passing the 

message signal m{i) through a new filter whose transfer function is related to that of the filter in 
Fig. la as follows: 



H Q (f) = j[H(f - f c ) - H(f + f c )], -W<f<W 



(14) 



Let m'(t) denote the output of this filter produced in response to the input m(t). Hence, we may 
express the quadrature component of the modulated signal s(t) as 



Accordingly, substituting Eqs. (11) and (15) in (8), we find that s(t) may be written in the 
canonical form 



There are two important points to note here: 

1. The in-phase component sj(t) is completely independent of the transfer function H(f) of the 

band-pass filter involved in the generation of the modulated wave s(t) in Fig. la, so long as it 
satisfies the condition of Eq. (6). 

2. The spectral modification attributed to the transfer function H(f) is confined solely to the 
quadrature component £g(0- 

The role of the quadrature component is merely to interfere with the in-phase component, so as to 
reduce or eliminate power in one of the sidebands of the modulated signal s(t), depending on the 
application of interest. 




(15) 




(16) 
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Problem 2.21 

(a) Expanding s(t), we get 

s(t) = 1 a A A cos(2irf t) oos(2irf t) 
2 m c c m 

1 1 

- A A sin(2Tif t) sin(2Tif t) + ^Cl-a) A A cos(2nf t) cos(2nf t) 
d m c c m 2 cm c m 

+ 5-0-a> A m A sin(2rf t) sin(2nf t) 

2 m c c m 

s r A A cos(2nf t) cos(2nf t) 
d m c c m 

+ i A A (1-2a) sin(2irf t) sin(2irft) 
d m c c in 

Therefore, the quadrature component is: 

_ ±(\k(1-2a) sin(2rft) 
2 c m m 



(b) After adding the carrier, the signal will be: 

s(t) = A [1 + 1 A cos(2Tif t)] cos(2Tif t) 
c d m m c 

+ 1a A (1-2a) sin(2nf t) sin(2nf t) 
d c m m c 

The envelope equals 

a(t) = A c / [1 + ^ A m cos(2Trf m t)] 2 + [1 A m (1-2a) sin (2nf t) ] 2 



1 / I 5" A J 1 -2 a) sin(2rft) 

s A [1 + i A cos(2 w f t)] /1 +' " m 



c 2 m m ./ , 1 . „ 

1 + 5- A m cos(2nf t) 
^_ dm m 

= A [1 + 1 A m cos(2nf t)] d(t) 
c dm m 

where d(t) is the distortion, defined by 



d(t) = /1 + 



r \ A(1-2a) sin(2Tif t) 
d m m 

1 + \ A cos(2uf t) 
2 m m 
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(c) d(t) is greatest when a = 0. 



Problem 2.22 



Consider an incoming narrow-band signal of bandwidth 10 kHz, and mid-band frequency 
which may lie in the range 0.535-1.605 MHz. It is required to translate this signal to a fixed 
frequency band centered at 0.455 MHz. The problem is to determine the range of tuning that 
must be provided in the local oscillator. 

Let f c denote the mid-band frequency of the incoming signal, and f t denote the local oscil- 
lator frequency. Then we may write 

0.535 <f c < 1.605 

and 

/c-/i = 0.455 

where both f c and /, are expressed in MHz. That is, 

/i=/c-0.455 

When / c = 0.535 MHz, we get /, = 0.08 MHz; and when f c = 1.605 MHz, we get /,= 1.15 MHz. 
Thus the required range of tuning of the local oscillator is 0.08-1.15 MHz. 
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Problem 2.23 

Let s(t) denote the multiplier output, as shown by 

s(t) = A g(t) cos(2Trf c t) 

where f, lies in the range f 0 to f Q -tW. The amplitude spectra of s(t) and g(t) are related 
as follows : 




|s(f)| 



\ a|g(o) 




f 
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With v(t) denoting the band-pass filter output, we thus find that the Fourier transform of 
v(t) is approximately given by 



V(f) * 1 A G(f c -f Q ) 



1 0 2 - 1 1 - 0 2 



The rms meter output is therefore (by using Rayleigh's energy theorem) 

oo 

2 1/2 
V„ m . = [/ v (t)dt]'^ 
rms „ 



[/ IV (f) I 2 df] 1/2 



[2(1 A 2 |G(f c -f D l 2 > Af] 1/2 



- |G(f -f )| /Af 
n ° ° 



Problem 2.24 

For the PM case, 

s(t) = A cos[2nf t + k m(t)]. 
c c p 

The angle equals 

6, (t) = 2irf t + k m(t). 
j. c p 

The instantaneous frequency, 

Ak Ak 

f i (t) = f c + 2^T-- Z 2^ 6(t - "V- 
0 n 



is equal to f, + Ak p /2TrT Q except for the instants that the message signal has 
discontinuities. At these instants, the phase shifts by -k p A/T Q radians. 



set") 
A. 










-A - 
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For the FM case, f^t) 



f e + k f m(t) 




Problem 2.25 



^■l-pUk-t- 



The instantaneous frequency of the mixer is as shown below: 



4Af -f 




The presence of negative frequency merely indicates that the phasor representing the 
difference frequency at the mixer output has reversed its direction of rotation. 

Let N denote the number of beat cycles in one period. Then, noting that N is equal 
to the shaded area shown above, we deduce that 

N = 2[4Af.f 0 x(^~ - t) + 2Af.f 0 x 2 ] 

= 4Af«x(1 - f 0 x) 
Since f Q x « 1, we have 
N a 4Af»x 

Therefore, the number of beat cycles counted over one second is equal to 

T7fT = %f 'V' 



86 



Problem 2.26 



The instantaneous frequency of the modulated wave s(t) is as shown below: 




T 
2 



T 
2 



We may thus express s(t) as follows 



cos(2rrf c t) , 



t < - 



s(t) = \ cos[2Tt(f +Af)t], 
c 



cos[2Ttf t), 
c 



T T 

T 

2< fc 



The Fourier transform of s(t) is therefore 



-T/2 

S(f) = / cos(2irf t) exp(-j2irft) dt 

c 



T/2 

+ / cos[2Ti(f +Af)t] exp(-j2irft) dt 
-T/2 



+ / cos(2Trf t) exp(-j2irft) dt 
T/2 C 



/ cos(2uf t) exp(-j2Trft) dt 



T/2 

+ / {cos[2Tr(f +Af)t - cos(2irf t)} exp(-j2Trft) dt 
-T/2 c c 



(1) 
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The second term of Eq. (1) is recognized as the difference between the Fourier transforms 

of two RF pulses of unit amplitude, one having a frequency equal to f +Af and the other 

having a frequency equal to f . Hence, assuming that f T » 1, we ma°y express S(f) as 
follows : c 

\ <5(f-f c ) + | sinc[T(f-f c -Af)] - | sinc[T(f-f c )], f > 0 




j 6(f+f Q ) + | sinc[T(f+f c +Af)] - | sinc[T(f+f c )], f < 0 

Problem 2.27 

For SSB modulation, the modulated wave is 



A 

s(t) = [m(t) cos(2nf t) ± fh(t) sin(2Trf t)], 
^ c c 

the minus sign applying when transmitting the upper sideband and the plus sign applying 
when transmitting the lower one. 

Regardless of the sign, the envelope is 

a(t) = ^ J m 2 (t) + m 2 (t) . 

(a) For upper sideband transmission, the angle, 

6(t) — 2irf t + tan~ 1 (S££|) . 
i c m(.t; 

The instantaneous frequency is, 

f .(t) = j- d ^t) 

2tt dt 

_ f + m(t) m'(t) - m(t) m f (t) 
° 2Tr(m 2 (t) + m 2 (t)) 

where 1 denotes time derivative. 

(b) For lower sideband transmission, we have 



and 



6.(t) = 2,f c t + tan" 1 (-§£>) , 



f.(t) = f , m'(t) - m(t) m'(t) 

1 C 2rr(m 2 (t) + m 2 (t)) 
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Problem 2.28 9 

(a) The envelope of the FM wave s(t) is 

a(t) = A c / 1+b 2 sin 2 (2irf t) 
The maximum value of the envelope is 

a = A /~ 1+g 2 
max c 

and its minimum value is 

a . = A 

mm c 

Therefore , 

max / . 2 
- — = / 1+3 
min 

This ratio is shown plotted below for 0 < g < 0.3: 




I s 

(b) Expressing s(t) in terms of its frequency components: 

s(t) = A cos(2rrf t) + i B A cos[2n(f +f )t] - i M cos[2Tr(f -f )t] 
c c^c cm d c cm 
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The mean power of s(t) is therefore 



A 2 3 2 A 2 3 2 A 2 
D c 2 c _ c 



A ' « 2 

r (1 + f-> 



The mean power of the unmodulated carrier is 

A 2 
c 2 

Therefore , 

P 2 

-1= 1 + £- 
P 2 
c 

which is shown plotted below for 0 < g < 0. 3: 




(c) The angle e^t), expressed in terms of the in-phase component, s^Xt), and the 
quadrature component, s (t), is: 

6.(t) = 2.f e t ♦ tan 

= 2tt f t + tan -1 [gsin(2nf t)] 
c m 

—1 3 
Since tan (x) = x - x /3 + ... , 

8 3 3 

e.(t) * 2i:f t + gsin(2Trf t) - f- sin J (2irf t) 
i c mi m 

The harmonic distortion is the power ratio of the third and first harmonics: 



D h 



For B = 0.3, D h = 0.09% 

Problem 2.29 

(a) The phase-modulated wave is 

s(t) = A cos[2irf t + k A cos(2rrf t)] 
c c p m m 

= A„ cos[27rf t + 8 cos(2irf t)] 
c c p m 

= A Q cos(2irf c t) cos[B p cos(2Trf m t)] - A Q sin(2irf c t) sin[g p cos(2Trf m t)] (1) 
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If 0 p < 0.5, then 

cos[ft cos(2nf t)] « 1 
p m 

sin[e cos(2nf t)] = B cos(2Trf t) 
P m p m 

Hence, we may rewrite Eq. (1) as 

s(t) = A cos(2Trf t) - B k n sin(2irf t) cos(2Trf t) 
v- i» p c c m 

= A c cos(27rf c t) - 1 g p A(; sin[2*(f c+ f m )t] 
-^g p A c 5in[2 1 r(f c -f in )t] 
The spectrum of s(t) is therefore 

S(f) = 1 A [5 (f-f ) + 6(f+f )] 

c. C C C 

-jj3 p A c [6(f-f c -f m ) -6(f + f c+ f B )]- 

-5j B p A c [6(f - f c + V - 6(f+ V f m )] 
(b) The phasor diagram for s(t) is deduced from Eq. (2) to be as follows: 

Lower side- frequency 




side- frequency 



The corresponding phasor diagram for the narrow-band FM wave is as follows: 
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" m Upper 
^ ■ _ side-frequency 



Lower side-frequency 



Comparing these two phasor diagrams, we see that, except „ for a phase difference, the 
narrow-band PM and FM waves are of exactly the same form. 

Problem 2.30 

The phase-modulated wave is 

s(t) = A cos[2irf t + g cos(2irf t)] 
c c p m 

The complex envelope of s(t) is 

s(t) = A c exp[jS p cos(2Trf m t)] 
Expressing s(t) in the form of a complex Fourier series, we have 



s(t) = E c n exp( j2irnf m t) 



where 



1/2f 

c n = f / m s(t) exp(-J2imf t) dt 
n m 

m 



1/2f 

= A f / m exp[jS„ cos(2irft) - j2irnft] dt 
1/2f P m m 

m 



(1) 



Let 2nf t = tt/2 - <j>. 
m T 

Then, we may rewrite Eq. (1) as 

A -n/2 

c i mr 

c = - p- exp(- ^^p-) / expCjg sin(<))) + jn<J>] d<|» 
n a 3tt/2 p 
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The integrand is periodic with respect to <)> with a period of 2ir. Hence, we may rewrite 
this expression as 

c n = 2^ exp(- j1 j L ) / exp[je p sin(<|>) + jn<j>] d<f> 

"IT 

However, from the definition of the Bessel function of the first kind of order n, we have 
1 11 

J n (x) = 27 / exp(j x sin<|> - nj<f> ) d<|> 

-IT 

Therefore , 

c n = A c exp(- jfL) J_ n ( 3p ) 

We inay thus express the PM wave s(t) as 
s(t) = Re[s(t) exp( j2Trf c t)] 

= A c Re[ E J_ n (3 p ) exp(- exp( ^nf^) exp( j2Trf,t)] 



8 A c Z J -n ( V cos[2TT(f c+ nf m )t - 
n=-«o 

The band-pass filter only passes the carrier, the first upper side- frequency, and the 
first lower side-frequency, so that the resulting output is 



S o (t) = A c J 0 ( V c°s(2*f c t) + A c J^fy cos[2 1 r(f c+ f m )t - §] 

+ A c J 1 ( V oo-KkCVV* *P 

= A n J n ( &n> cos(2irf t) + A J ,(« ) sin[2Tr(f +f )t] 
cup c c — I p cm 



- A JA$) sin[2rr(f -f )t] 
c i p cm 



But 



J.! (B p ) = -J^gp) 
Therefore , 

S o (t) = A c J 0 (P p ) cos(2nf c t) 



- A J-(g_) {sin[2n(f +f )t] + sin[27r(f -f )t]} 
^ ' v cm cm 

A c J 0 ( V cos^^t) - 2 A Q J^Bp) cos(2nf m t) sin(2Trf c t) 



The envelope of s (t) equals 
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a(t) = / J^(g ) + 4J 2 (g ) cos 2 (2Trf t) 
cup i p m 

The phase of s (t) is 




The instantaneous frequency of s (t) is 



f^t) 



L. d<t>(t) 
2tt dt 



f + 



2 J 0 (3 p ) J^Bp) sin(2Trf m t) 



c 



J 0 ( V + 4J 1 ( V cos 2 (2 1 ;f m t) 



Problem 2.31 

(a) From Table A4. 1, we find (by interpolation) that J Q (g) is zero for 
3 = 2.44, 

3 = 5.52, 
3 = 8.65, 
3 = 11.8, 
and so on. 

(b) The modulation index is 

Q Af k f A m 

3 = T = ~7 

m m 

Therefore , 

k 51 

f A 

m 

Since J Q (3) = 0 for the first time when 3 = 2.44, we deduce that 



= 1.22 x 1(r hertz/volt 

Next, we note that J Q (B) = 0 for the second time when 3 = 5.52. Hence, the corresponding 
value of A for which the carrier component is reduced to zero is 



2.44 x 10 

2 



3 



k 



f 



95 



a m 



5.52 x 10- 
1.22 x 10 : 



4.52 volts 



Problem 2.32 



For 3 = 1, we have 
J Q (1) = 0.765 
J.,(1) = 0.44 
J 2 (D = 0.115 

Therefore, the band-pass filter output is (assuming a carrier amplitude of 1 volt) 

sAt) = 0.765 cos(2nf t) 
o c 

+ 0.44 {cos[2ir(f +f )t] - cos[2Tr(f -f )t]} 
cm cm 

+ 0.115 {cos[2Tt(f +2f )t] + cos[2Ti(f -2f )t]} , 
cm cm 

and the amplitude spectrum (for positive frequencies) is 



0.22 



0 



o.oSg 

_L_ 



0.22 



c <>r> c &> 



c 



O.05F 



c "m c "to 
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Problem 2.33 

(a) The frequency deviation is 

Af = k f A ffl = 25 x 10 3 x 20 = 5 x 10 5 Hz 
The corresponding value of the modulation index is 

a Af 5 x 10 5 c 

B = f~ = 5 — = 5 

m 10 5 

The transmission bandwidth of the FM wave, using Carson's rule, is therefore 
B T = 2f m (1+3) = 2x100 (1+5) = 1200 kHz = 1.2 MHz. 

(b) Using the universal curve of Fig. we find that for 3=5: 

Af " 3 

Therefore , 

B T = 3x500 = 1500 kHz =1.5 MHz 

(c) If the amplitude of the modulating wave is doubled, we find that 
Af = 1 MHz and 3 = 10 

Thus, using Carson's rule we obtain 

B T = 2x100 (1+10) = 2200 -kHz = 2. 2 MHz 
Using the universal curve of Fig. 3<3o, we get 

B T 

If - 2 ' 75 
and B T = 2.75 MHz. 

(d) If f m is doubled, g = 2.5. Then, using Carson's rule, B T = 1.4 MHz. Using the 
universal curve, B^Af = M, and 

B T = 4Af = 2 MHz. 
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Problem 2.34 

(a) The angle of the PM wave is 

QAt) = 2irf t + k m(t) 
i c p 

= 2rrf t + k A cos(2irf t) 
c p m m 

= 2irf t + g cos(2irf t) 
c p m 

where g p = k p A m . The instantaneous frequency of the PM wave is therefore 

, d6 (t) 

f i (t) =ir-^t- 

= f c " *p f m »in(2.f B t) 

We see that the maximum frequency deviation in a PM wave varies linearly with the 
modulation frequency f m . 

Using Carson's rule, we find that the transmission bandwidth of the PM wave is 
approximately (for the case when g p » 1) 

B_ = 2(f + g n f ) = 2f (1 + g ) = 2f g 
I m p m m p m p 

This shows that B T varies linearly with f m . 

(b) In an FM wave, the transmission bandwidth B T is approximately equal to 2Af, if the 

modulation index g » 1. Therefore, for an FM wave, B T is effectively independent of the 

modulation frequency f . 

m 

Problem 2.35 

The filter input is 
v^t) = g(t) s(t) 

= g(t) cos(2Trf c t - irkt 2 ) 

The complex envelope of v^(t) is 

~ 2 
v^t) = g(t) exp(-jirkt ) 

The ^impulse response h(t) of the filter is defined in terms of the complex impulse 
response h(t) as follows 



h(t) = Re[h(t) exp(j2Tif t)] 

c 



With 



h(t) = cos(2nf t + irkt 2 ), 
c 



we have 



h(t) = exp(jTTkt ) 0 



if -the 



The complex envelope 'filter output is therefore (see ft-ppe-mLi % <2.) 
v Q (t) =|hCt)^2r VjCt) 



1 2 2 

g(x) expC-jirkx ) exp[ jirk(t-x )] dx 



-00 

1 2 
= 2 ex P(jTikt ) / g(x) exp(-j2Trktx) dx 



^ expCjukt 2 ) G(kt) 



Hence , 



|v 0 (t)| = \ lG(kt)l 

This shows that the envelope of the filter output is, except for the scale factor of 1/2, 
.equal to the magnitude of the Fourier transform of the input signal g(t), with kt playing 
the role of frequency f. 



Problem 2.36 

The overall frequency multiplication ratio is 
n = 2x3 = 6 

Assume that the instantaneous frequency of the FM wave at the input of the first frequency 
multiplier is 

f i1 (t) = f c + Af cos ( 2 Tr f m t) 

The instantaneous frequency of the resulting FM wave at the output of the second frequency 
multiplier is therefore 

f._(t) = nf + nAf cos(2nf t) 
12 c m 

Thus, the frequency deviation of this FM wave is equal to 

nAf = 6x10 r 60 kHz 
and its modulation index is equal to 

nAf 60 1? 

f m = 5 
m 

The frequency separation of the adjacent side- frequencies of this FM wave is unchanged at 
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Problem 2.37 



(a) Figure 1 shows the simplified block diagram of a typical FM transmitter (based on the indirect 
method) used to transmit audio signals containing frequencies in the range 100 Hz to 15 kHz. The 
narrow-band phase modulator is supplied with a carrier signal of frequency fa = 0.2 MHz by a 
crystal-controlled oscillator. The desired FM signal at the transmitter output is to have a carrier 
frequency f c = 100 MHz and a minimum frequency deviation Af= 75 kHz. 

In order to limit the harmonic distortion produced by the narrow-band phase modulator, we 
restrict the modulation index pi of this modulator to a maximum value of 0.3 radians. Consider 
then the value = 0.2 radians, which certainly satisfies this requirement. The lowest modulation 
frequencies of 100 Hz produce a frequency deviation of Afa = 20 Hz at the narrow-band phase 
modulator output, whereas the highest modulation frequencies of 15 kHz produce a frequency 
deviation of Afa - 3 kHz. The lowest modulation frequencies are therefore of immediate concern, 
as they produce a much lower frequency deviation than the highest modulation frequencies. The 
requirement is therefore to ensure that the frequency deviation produced by the lowest modulation 
frequencies of 100 Hz is raised to 75 kHz. 



Baseband 
signal 



Integrator 



Narrowband 
phase 
modulator 



Frequency 
multiplier 



0.1 MHz 




Frequency 
multiplier 
n. 



FM signal 



9.5 MHz 



Crystal 
controlled 
oscillator 



Crystal- 
controlled 
oscillator 



Figure 1 

To produce a frequency deviation of Af = 75 kHz at the FM transmitter output, the use of 
frequency multiplication is obviously required. Specifically, with Afa = 20 Hz and A/= 75 kHz, 
we require a total frequency multiplication ratio of 3750. However, using a straight frequency 
multiplication equal to this value would produce a much higher carrier frequency at the 
transmitter output than the desired value of 100 MHz. To generate an FM signal having both the 
desired frequency deviation and carrier frequency, we therefore need to use a two-stage frequency 
multiplier with an intermediate stage of frequency translation as illustrated in Fig. 1. Let ni and n 2 
denote the respective frequency multiplication ratios, so that 

Af 75000 „ <n 



100 



The carrier frequency nrfy at the first frequency multiplier output is translated downward to (f 2 - 
ftj/i) by mixing it with a sinusoidal wave of frequency f 2 = 95 MHz, which is supplied by a 
second crystal-controlled oscillator. However, the carrier frequency at the input of the second 
frequency multiplier is required to equal / c /n 2 . Equating these two frequencies, we thus get 

fc 

fl- n \f\ = — 



Hence, with/ 2 = 0.1 MHz,/ 2 = 9.5 MHz, and/ c = 100 MHz, we have 

o < ft 1 100 
9.5-O.lnj = 



(2) 



Solving Eqs. (1) and (2) for n x and n 2 , we obtain 

n { = 75 
n 2 = 50 



(b) Using these frequency multiplication ratios, we get the set of values indicated in the table 
below: 



Table -Values of Carrier Frequency and Frequency Deviation at the 
Various Points in the Wide-band Frequency Modulator of Fig. 1 





At the Phase 
Modulator 
Output 


At the First 
Frequency 
Multiplier 
Output 


At the Mixer 
Output 


At the Second 
Frequency 
Multiplier 
Output 


Carrier 
frequency 


0.1 MHz 


7.5 MHz 


2.0 MHz 


100 MHz 


Frequency 
deviation 


20 Hz 


1.5 kHz 


1.5 kHz 


75 kHz 
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Problem 2.38 



(a) Let L denote the inductive component, C the capacitive component, and C Q the 
capacitance of each varactor diode due to the bias voltage acting alone. Then, we have 

C 0 = 100 V" 1/2 P F 

and the corresponding frequency of oscillation is 

1 



f 0 



2irr0Xc+C o /2) 



Therefore , 
10 6 = 



2tt/200 x 10" 6 (100 x 10" 12 + 50 V~ 1/2 x 10 -12 ) 

b 



Solving for V b , we get 
V fe = 3.52 volts 



(b) The frequency multiplication ratio is 64. Tnerefore, the modulation index of the FM 
wave at the frequency multiplier input is 

3 = fif = 0.078 

This indicates that the FM wave produced by the combination of L, C and the varactor 
diodes is a narrow-band one, which in turn means that the amplitude of the modulating 
wave is small compared to V b . We may thus express the instantaneous frequency of this FM 
wave as follows : 

f. (t) = i- [200 x 10" 6 {100 x 10" 12 + 50 x 10~ 12 [3.52 + A sin(2irf t) ]" 1/2 }]"" 1/2 
i 2n mm 

= 12—- {1 + 0.266 [1 + t^§~ sin(2Ttf t)]" l/2 }" 1/2 
2/2ir 3,52 m 

7 A 
« ^— {1 + 0.266 [1 - ^-jSjT sin(2Trf t)]}" 1/2 
2/2* 7 * 04 m 

= 10 6 [1 - 0.03 A sin(2iif t)]" 1/2 
m m 

= 10 6 [1 + 0.015 k sin(2nf t)] 
m m 
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With a modulation index of 0.078, the corresponding value of the frequency deviation 

is 



Af = 3 f 
w m 



= 0.078 x 10 1 * Hz 

Therefore , 

0.015 A x 10 6 = 0.078 x -10^ 
m 

where A is in volts. Solving for A . we get 
m m ° 

A m = 52 x 10" 3 volts, 
m 

Problem 2.39 

The transfer function of the RC filter is 
H(r\ - jSirfCR 

If 2-irfCR « 1 for all frequencies of interest, then we may approximate H(f) as 
H(f) = j2nfCR 

However, multiplication by j2irf in the frequency domain is equivalent to differentiation 
in the time domain. Therefore, denoting the RC filter output as v.,(t), we may write 

v l( t) =CR^) 

ri t 
= CR XT {A cos[2nf t + 2%. / m(t) dt]} 
dt c c f q 

t 

= -CR A [2wf + 2nk.m(t)] sin[2Trf t + 2irk» / m(t) dt] 
c c f c 0 

The corresponding envelope detector output is 

k f 

v 0 (t) « 2uf CR A |1 + 7 1 m(t) 
2 C C I r 

c 

Since k f (m(t)j < f for all t, then 

k f 

v 0 (t) * 2irf CR A [1 + j~- m(t)] 

d C C I 

c 

which shows that, except for a dc bias, the output is proportional to the modulating 
signal m(t) . 
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Problem 2.40 

The envelope detector input is 

v(t) = s(t) - s(t-T) 

= A c cos[2irf c t+ <j,(t)] - A q cos[2nf (t-T ) + +(t-T)] 

2irf (2t-T) + <(>(t) + <()(t-T) 2*f T + <|>(t) - <j>(t-T) 
= -2A c sin[ 2 _ -, sint c _ ] 

where 

<|>(t) = g sin(2Trf m t) 

The phase difference <j>(t) - <j>(t-T) is 

<J»(t) - <|>(t-T) = B sin(2Tif t) - g sin[2rf (t-T)] 

m m 

= g[sin(2irf t) - sin(2irf t) cos(2irf T) + cos(2Trf t) sin(2Trf T)] 
m mm mm 

* 3[sin(2nf t) - sin(2nf t) + 2itf T cos(2irf t)] 
m m m m 

= 2irAfT cos(2Trf t) 
m 

where 

Af = gf m . 

Therefore, noting that 2nf T = tt/2, we may write 

c 

2irf T + <|>(t) - <|>(t-T) 

sin[ 5 ] = sin[irf T + irAfT cos(2irf t)] 

£ cm 

__ = sin[£ + irAfT cos(2uf t)] 

= /2 cos[irAfT cos(2irf t)] W2 sin[nAfT cos(2nf t)] 

m m 

= /2 + /5 irAfT cos(2TTf t) 

m 

where we have made use of the fact that irAfT « 1. We may therefore rewrite Eq. (1) 

v(t) » -2/5 A c [1 + ir Af T 005(2^)] sin[ Trf, (2t-T) + 4» (t) + 2 ^ t - T > ] 

Accordingly, the envelope detector output is 

a(t) =2/2 A [ 1 + trAfT cos(2Trf t) ] 
c m 

which, except for a bias term, is proportional to the modulating wave. 
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Problem 2.41 

(a) In the time interval t-CT.,/2) to t+(T 1 /2), assume there are n zero crossings. The 
phase difference is e^t+^/2) - 6 i (t-T 1 /2) = nir. Also, the angle of an FM wave is 

t 

6. (t) — 2 iff t + 2irk~ / m(t) dt . 
1 ° f 0 

Since m(t) is assumed constant, equal to m.,, 6 i (t) = 2irf c t + 2ik f m.|t. Therefore, 

6 i (t+T 1 /2) - 6 i (t-T 1 /2) = (2Trf c + [t+T 1 /2 - (t-T.,/2)]. 

= (2irf + 2irk-m.,) T,. 
c ill 

But 

d6 (t) 

f i (t) = -dt" = 2ltf c + 2 *f»1 • 

Thus, 

e i (t+T 1 /2) - e i (t-T 1 /2) = f t (t) T r 
But this phase difference also equals nir. So, 
fj^t) T 1 = nir 

and 

f^Ct) = nir/T 1 

(b) For a repetitive ramp OS the modulating wave, we have the following set of waveforms 
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m(t) 



0 

s(t) 




Limiter 
output 



Pulse 

generator 

output 



Lour, pees 



Problem 2.42 



The complex envelope of the modulated wave s(t) is 
s(t) = a(t) exp[j<|>(t)3 

Since a(t) is slowly varying compared to expC j<|>(t) ] , the complex envelope s(t) is 
restricted effectively to the frequency band - B T /2 <_ f < An ideal frequency 

discriminator consists of a differentiator followed by an envelope detector. The output 
of the differentiator, in response to s(t), is 

».<« ■ at J<t) 

= |j {a(t) expect)]) 



= ^fj^ exp[Jt(t)3 + j a(t) exp[j$(t)] 



= a 



Since a(t) is slowly varying compared to <(>(t), we have 
|d<t>(t)| ss J da(t) 



\2M±l\ ss | J da(t) t 

1 dt ■ " l a (t) ~dt ' 



Accordingly, we may approximate v Q (t) as 



v o (t) - j a(t) ^1 exp[j*(t)] 



However, by definition 



<(>(t) r -2nk, / m(t) dt 
0 



Therefore , 

v Q (t) = j2Trk f a(t) m(t) exp[j<j)(t)] 
Hence, the envelope detector output is proportional to a(t) m(t) as shown by 

|v Q (t) | * 2Trk f a(t) m(t) 

Problem 2.43 

(a) The limiter output is 



z(t) = sgn{a(t) cos[2Tff c t + <}>(t)]} 
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Since a(t) is of positive amplitude, we have 
z(t) = sgn{cos[2Trf c t + <))(t)]} 

Let 

i|>(t) = 2irf c t + $(.t) 
Then, we may write 



sgn[cos ip] = I c R exp(jn\|>) 
n=— oo 



1 * 

c n = 2^ / sgnCcos exp(-jni{0 di|» 



1 -*/2 ir/2 



S (-1) exp(-jn^)d^ + p- / (+1) exp(-jn^) dt|> (1) 
-tt -ir/2 



1 ' 

+ ~- / (-D exp(-jn^) dip 
ir/2 

If n / 0, then 

c n = 2Tr(-jn) C- e xP(^)+exp(jnTr)+exp( : ^)-exp(^)-exp(-jnTr)+exp(=^-)] 

= lj-[2 8in(£|)-sin(nir)] 

2 , ..(n-D/2 

—(-1 ) , n odd 

irn 

0, n even 

If n=0, we find from Eq. (1) that c n =0. Therefore, 

oo 

sgnCcos i|>] = f I £ (-1) (n " 1)/2 exp(jn^) 
n= -» 
n odd 

4 " (-1) k 
= ^ k= Z 0 kn- eos[*(2k + 1)] 

We may thus express the limiter output as 
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ZU) = ^ k^O COS[27rf c t(2k+1) + ♦<t)(2k+1)] (2) 

(b) Consider the term 

cos[2Trf c t(2k+1 ) + 4>(t) (2k+1)] = Re{exp[j2Trf c t(2k+1)]exp[j<j)(t)(2k+1)]} 

* Re{exp[j2Trf c t(2k+1)][exp(j<f>(t))] 2k+1 } 
The function expt j<Kt) ], representing the complex envelope of the FM wave with unit 
amplitude, is effectively low-pass in nature. Therefore, this term represents a band-pass 
signal centered about ±f c (2k+1). Furthermore, the Fourier transform of {exp[ j(j>(t) ]} 214+1 
is equal to that of exp[j<}>(t)] convolved with itself 2k times. Therefore, assuming that 
exp[j<j>(t)] is limited to the interval -Bj/2 < f < Bj/2, we find that (exp[ j<j>(t) ]) is 
limited to the interval -(B T /2) (2k+1 ) < f < (B T /2) (2k+1 ) . 

Assuming that f Q > B^., as is usually the case, we find that none of the terms 

corresponding to values of k greater than zero will overlap the spectrum of the term 

corresponding to k=0. Thus, if the limiter output is applied to a band-pass filter of 

bandwidth B^, and mid-band frequency f , all terms, except the term corresponding to k=0 in 

Eq. (2), are removed by the filter. The resulting filter output is therefore 

y(t) = - cos[2Trf t + *(t)] 
ir c Y 

We thus see that by using the amplitude limiter followed by a band-pass filter, the effect 
of amplitude variation, represented by a(t) in the modulated wave s(t), is completely 
removed. 

Problem 2.44 

(a) Let the FM wave be defined by 

t 

s(t) = A eos[2i:f t + 2irk- / m(t) dt] 
c c o 

Assuming that f Q is large compared to the bandwidth of s(t), we may express the complex 
envelope of s(t) as 

t 

s(t) = A exp[j2nk f . / m(t) dt] 
° 0 

But, by definition, the pre-envelope of s(t) is fif^rijiz X) 
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s + (t) = s(t) exp(j2Trf c t) 
= s(t) + j s(t) 
where S(t) is the Hilbert transform of s(t). Therefore, 

t 

s(t) + j§(t) = A q exp[j2Trk f / m(t) dt] exp(j2Trf t) 



t t 
= A {cos[2Trf t + 2Trk- / m(t) dt] + j sinl^r t + 2irk, f m(t) dt]} 
. ° 1 0 C f 0 

Equating real and imaginary parts, we deduce that 

t 

S(t) = A c sin[2Ttf c t + 2tk f f mCt) dt] (1) 

(b) For the case of sinusoidal modulation, we have 

m(t) = A m cos(2nf t) 
m m 

The corresponding FM wave is 



s(t) = A c cos[2nf c t + g sin(2irf m t)] 



where 

i m 

Expanding s(t) in the form of a Fourier series, we get 

00 

s(t) = A Z J (3) cos[2ir(f +nf )t] 

C n= _oo " C m 

Noting that the Hilbert transform of cos[2if(f +nf )t] is equal to sin[2ir(f +nf )t], and 
using the linearity property of the Hilbert transform, we find that the Hilbert transform 
of s(t) is 

00 

s(t) = A o Z J (3) sin[2ir(f +nf ) t] 
n=-°o cm 

= A c sin[2iff c t + 3 sin(2iTf m t)] 

This is exactly the same result as that obtained by using Eq. (1). In the case of 
sinusoidal modulation, therefore, there is no error involved in using Eq. (1) to evaluate 
the Hilbert transform of the corresponding FM wave. 

Problem 2.45 

(a) The modulated wave s(t) is 

s(t) = exp[-4(t)] cos[2itf t + <f>(t)] 

c 1 10 



= Re{exp[-^(t)] exp[j2Ttf c t + j<f>(t)]} 
= Re{exp[ j2Trf c t + j( 4>(t) + j^(t))]} 

= Re{exp[ j2Trf c t + j<f, + (t)]} (1) 
where 4> + (t) is the pre-envelope of the phase function <j>(t), that is, 
<(> + (t) = <j>(t) + j^(t) 

Expanding the exponential function exp[ j<f> + (t) ] in the form of an infinite series: 

00 .n 

exp[j<|> + (t)] = Z ij- 4>J(t) (2) 
n=0 

Taking the Fourier transform of both sides of this relation, we may write 

» .n 

; FCexpCj^U)]} = Z F[<|>"(t)3 

n=0 

For n>2, we may express <|>"(t) as the product of <|> + (t) and <j>"~ 1 (t). Hence, 

F[^(t)] = * + (f)^-F[^- 1 (t)] 

where <|) + (t) ^ * + (f), and T^- denotes convolution. Since * + (f) = 0 for f < 0, it follows 
that for all n ^ 0, 

F[$"(t)3 =0, for f < 0 

Hence , 

F{exp[ j<|> + (t)]} = 0 for f < 0 
By using the frequency-shifting property of the Fourier transform, it follows that 

F{exp[j<J> (t)] exp(j2Trf t)} = 0 for f < f (3) 
■ c c 

From Eq. ( 1 ) , 

s(t) = -1 {exp[j2Trf c t + j(|> + (t)] + exp[-j2Trf c t - j<j,*(t)]} 
where <|> + (t) is the complex conjugate of <() + (t). Therefore, 

F[s(t)] = | F{exp[j2Trf c t + j<t> + (t)]} + j F{exp[-j2Trf Q t - j<t>*(t) } 

Applying the conjugate-function property of the Fourier transform to Eq. (3), we find that 

F{exp[-j2irf t - jAt)]} = 0, for f > -f-. 

c + c 
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Hence, it follows that the spectrum of s(t) is zero for -f < f < f . However, this 

c c 

spectrum is of infinite extent, because the expansion of s(t) contains an infinite number 
of terms, as in eq. (2). 

(b) With 

<()(t) = g sin(2nf m t), 

we find that 

$(t) = -g COS(2irf t) 



m 



Therefore , 



♦ + (t) 



= g sin(2irf t) - jg cos(2irf t) 



- jg[cos(2Trf t) + j sin(2irf t)] 



= - jg exp(j2Trft) 



Hence , 



exp[ j<j) + (t)] = exp[g exp(j2Trf t)] 




The modulated wave s(t) is therefore 



s(t) = Re{exp( j2Tif c t) exp[ j$ + (t) ] } 

OS fj 

= Re[exp(j2irf t) I exp( j2irnf t)] 



n=0 



= Re{ Z ^7 exp[j27r(f c +nf m )t]} 



n=0 



00 



= Z 
n=0 



n! 



cos[2ir(f +nf )t] 



c m 
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Problem 2.46 



After passing the received signal through a narrow-band filter of bandwidth 8kHz centered on 
f c = 200kHz, we get 

x(t) = A c m(t) cos (2nf J) + n'(t) 

= A c m(t)cos(2nf c t) + n' 7 (Ocos(27i/ c O - n , / (Osin(27i/ c /) 

= (A c m(t) + n z (0)cos(27t/ c O - n' Q (t)sm(2nf c t) 

where n'(t) is the narrow-band noise produced at the filter output, and n' T (t) and n' Q (t) are its 
in-phase and quadrature components. Coherent detection of x(t) yields the output 

y(t) = A c m(t) + n' j(t) 
The average power of the modulated wave is 

a]p 

-4- = iow 

4 

where P is the average power of m(t). To calculate the average power of the in-phase noise 
component n t (t) , we refer to the spectra shown in Fig. 1: 

• Part (a) of Fig. 1 shows the power spectral density of the noise n(f), and a superposition of the 
frequency response of the narrow-band filter. 

• Part (b) shows the power spectral density of the noise n'j(t) produced at the filter output. 

• Part (c) shows the power spectral density of the in-phase component n'j(t) of n'{t) . 

Note that since the bandwidth of the filter is small compared to the carrier frequency/,,, we have 
approximated the spectral characteristic of n'{t) to be flat at the level of 0.5 x 10 6 watts/Hz. 
Hence, the average power of n' 7 (r) is (from Fig. lc): 

(10" 6 watts/Hz) (8 x 10 3 ) = 0.008 watts 
The output signal-to-noise ratio (SNR) is therefore 

Expressing this result in decibels, we have an output SNR of 31 dB. 



113 




! i i 




I ! I 



! I 



I ! ! 
i S i 



4( it-tftj 



1 SAU 




H 




(tJaAJL:/ H z ) 


"1 ! i ! I 1 1 1 ! 


I i ' V i i i ' i 

I i M M 
-j i — i — i * 


i 1 i i i i 

M ! 1 ! i 
i=4_^ — 1 — j — j — | — 




i i j | 

1 1 ! 1 

i ! ! I 
Mi! 



I i 



■fCk H*) 



114 



Problem 2.47 

autocorrelation 5 '^' ™ ^ qUadrature components of a narrow-band noise have 



R N (t) = R N (t) = R N (t) cos(2nf t) + R n (t) sin(2TTf c x) 
where R N (x) is the autocorrelation of the narrow-band noise, R w (x) is the Hilbert 



N 



transform of R N (x), and f is the band center. The cross-correlations of the quadrature 
components are 



R N N (x) = -R N N (x ) r R N (x) sin(2nf c x) - R N (x) cos(2irf c x) 

(a) For a DSBSC system, 

R N (t) = R N (t) = R m (t) cos(2irf„x) + R M (x) sin(2Trf„x) 



Q 



'IT 



N 



R N N (x) = " R N N (t) = R N (t) s i n(2irf (J t) - R N (x) cos(2Trf c x) 



r Q 



where f is the carrier frequency, and R^Cx) is the autocorrelation function of the 

narrow-band noise on the interval f-W<f<f+W. 

c — — c 



(b) For an SSB system using the lower sideband, 



R M (x) = R M (x) = R M (x) cos(2ir(f c - |) T ) + R u (x) sin(2Tr(f^- £)t) 



5j 



c 2' 



R N N (T) = ~ R N N (t) = R N (T) sin(2ir(f c - |)x) - R n (t) cos(2ir(f c - |)x) 

where in this case, R N (x) is the autocorrelation of the narrow-band noise on the interval 

f - W < f < f . 

c — — c 

(c) For an SSB system with only the upper sideband transmitted, the correlations are 

W W 
similar. to (b) above, except that (f - •=•) is replaced by (f + ■=■), and the narrow-band 

c d. c d 

noise is on the interval f < f < f + W. 

c — — c 

Problem 2.48 



c(t) 


Band-pass 






filter 


■< 




v(t) 



Low-pass 
filter 



y(t) 



cos[2iTf t+6 (t) ] 
c 
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The signal at the mixer input is equal to s(t) + n(t), where s(t) is the modulated wave, and n(t) is 
defined by 

n(t) = n j{t) cos (2Tif c t) - n Q (t) sin (2nf c t) 
with 

E[n){t)} = E[n 2 Q (t)] = N Q B T 

The s(t) is defined by for DSB-SC modulation 

s(t) = A c m(t) cos (2nf c t) 

The mixer output is 

v(0 = [j(0 + n(0]cos[27t/ c r + 8(/)] 

= { [A c m(t) + nj(t) cos(27i f c t) - n Q (t) sin(27t f c t) }cos2tt f c t + 9(0] 

= \[A c m(t) + n,{t){ cos [9(0] } + cos[47i/ c f + 9(0] 

+ \A c n Q (t){ sin[9(0] - sin[47t/ c r + 9(0] } 

The postdetection low-pass filter removes the high frequency components of v(t), producing the 
output 

y(0 = ^[[A c m(0 + n 7 (0]cos[9(0] + ^A c n e (0sin[9(0] (1) 

When the phase error 9(0 is zero, we find that the message signal component of the receiver 
output is ^A c m(0 • The error at the receiver output is therefore 

A c 

e{t) = y(0-ym(0 

The mean-square value of this error is 

e = E[e 2 (t)] 
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= E 



A c 

y{t)--£m{t) 



(2) 



Substituting Eq. (1) into (2), expanding the expectation, and noting that the processes m(t), 0(0, 
nj(i) and ng(t) are all independent of one another, we get 



e = ^E[m 2 (t)]E[(cos 2 Q(t))] + ^E[n 2 (t)]E[cos 2 Q(t)] 



+ ±E[n 2 Q (t)]E[sm 2 Q(t)] 



+ ^E[m 2 (t)] - ^E[m 2 (t)]E[cosQ(t)] 



We now note that 



E[n 2 (t)] = E[n 2 Q (t)] = a N 



E[n 2 (t)]E[cos\t)] + E[n 2 Q (t)]E[sm 2 Q(t)] 



= a 



N 



Therefore, 



e = -^E[m 2 (t)]E{[l-cosQ(t)] 2 } + ^ 



A l p 2 <*t 

-^-£{[l + cos9(f)] } + -^ 



where P = E[m (t)]. 

For small values of 0(r), we may use the approximation 



1 - cos0(r) ~ 



N 



Hence, 
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A 2 C P 4 al 

2 

Since 0(0 is Gaussian-distributed with zero mean and variance o e , we have 
E[Q\t)] = 3a* 

The mean-square error for the case of a DSBSC system is therefore 
16 + T 
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Problem 2.49 



Consider the case of a receiver using coherent detection, with an incoming single-sideband (SSB) 
modulated wave. We assume that only the lower sideband is transmitted, so that we can express 
the modulated wave as 

s(t) = ^CA c cos(2nf c t)m(t) + ^CA c sm(2nf c t)m(t) (1) 

where m(t) is the Hilbert transform of the message signal m{t). The system-dependent scaling 
factor C is included to make the signal component s(t) have the same units as the noise component 
n{t). We may make the following observations concerning the in-phase and quadrature 
components of s(t) in Eq. (1): 

1. The two components m(t) and m(t) are orthogonal to each other. Therefore, with the message 
signal m(t) assumed to have zero mean, which is a reasonable assumption to make, it follows 
that m(t) and m(t) are uncorrected; hence, their power spectral densities are additive. 

2. The Hilbert transform m(t) is obtained by passing m{t) through a linear filter with a transfer 
function - jsgn(f). The squared magnitude of this transfer function is equal to one for all /. 
Accordingly, we find that both m(i) and m{t) have the same power spectral density. 

Thus, using a procedure similar to that in Section 2.11, we find that the in-phase and quadrature 

2 2 

components of the modulated signal s(t) contribute an average power of C A C P/S each, where P 

2 2 

is the average power of the message signal m(t). The average power of s(t) is therefore C A c P/4 . 
This result is half that in the DSB-SC receiver, which is intuitively satisfying. 

The average noise power in the message bandwidth W is WN 0 , as in the DSB-SC receiver. Thus 
the channel signal-to-noise ratio of a coherent receiver with SSB modulation is 

2 2 

C A Z P 

(SNR)c,ssb = ^ (2) 

As illustrated in Fig. la, in an SSB system the transmission bandwidth B T is W and the mid-band 
frequency of the power spectral density of the narrow-band noise n(t) is offset from the 
carrier frequency f c by W/2. Therefore, we may express n(t) as 



n(t) = n z (r)cos 



sin 



(3) 



The output of the coherent detector, due to the combined influence of the modulated signal s(t) 
and noise n(f), is thus given by 
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y(t) = ^CA c m(t) + ^Jij(t) cos (71 Wt) + ^n Q (t)sm(nWt) (4) 

As expected, we see that the quadrature component m(t) of the modulated message signal s(t) 
has been eliminated from the detector output, but unlike the case of DSB-SC modulation, the 
quadrature component of the narrow-band noise n{t) now appears at the receiver output. 

The message component in the receiver output is CA c m{t)IA, and so we may express the average 

2 2 

power of the recovered message signal as C A c P/\6. The noise component in the receiver 
output is [rc 7 (7)cos(7iM) + ng(7)sin(7iW)]/2. To determine the average power of the output noise, 
we note the following: 

1. The power spectral density of both n^t) and ng(t) is as shown in Fig. lb. 

2. The sinusoidal wave cos(nWt) is independent of both n^t) and nqif). Hence, the power 
spectral density of n'j(t) = n[(t)cos(nWt) is obtained by shifting S N (f) to the left by 

W/2, shifting it to the right by W/2, adding the shifted spectra, and dividing the result by 4. 
The power spectral density of n' Q (t) = n Q (t) sin (nWt) is obtained in a similar way. The 

power spectral density of both n'j(t) and n Q {t) , obtained in this manner, is shown sketched 
in Fig. lc. 

From Fig. lc we see that the average power of the noise component n \{t) or n ' Q (t) is WNq/2. 
Therefore from Eq. (4), the average output noise power is WNq/4. We thus find that the output 

signal-to-noise ratio of a system, using SSB modulation in the transmitter and coherent detection 
in the receiver, is given by 

2 2 
CAP 

(SNR)o.ssb = ~ (5) 

Hence, from Eqs. (2) and (5), the figure of merit of such a system is 
(SNR) 0 



(SNR) C 



= 1 (6) 

SSB 



where again we see that the factor C cancels out. 

Comparing Eqs. (5) and (6) with the corresponding results for DSB-SC modulation, we conclude 
that for the same average transmitted ( or modulated message) signal power and the same average 
noise power in the message bandwidth, an SSB receiver will have exactly the same output signal- 
to-noise ratio as a DSB-SC receiver, when both receivers use coherent detection for the recovery 
of the message signal. Furthermore, in both cases, the noise performance of the receiver is the 
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as that obtained by simply transmitting the message signal itself in the presence of the same noise. 
The only effect of the modulation process is to translate the message signal to a different 
frequency band to facilitate its transmission over a band-pass channel. 
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Problem 2.50 



The power spectral density of the message signal m(t) i 



s as follows 



S (f) 
M 




-W 0 W 

The average signal power is therefore 



p = ; s M (f>df 



w f 

= 2 / a 1 df 
0 

= aW 

The corresponding value of the output signal-to-noise ratio of the SSB receiver is 
therefore, (using the solution to Problem 2 0 49) 



(SNR )_ = 



A 2 P 
c 



0 " 4WN 



0 

A 2 aW 
c 



a A 

w: 
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Problem 7 SI 

(a) If the probability 

P(ln s (t)| > eA c |1 + k a m(t) I) < 6 1 , 
then, with a probability greater than 1 - 6^, we may say that 
y(t) « {[A c + A Q k a m(t) + n c (t)] 2 } 1/2 

That is, the probability that the quadrature component n g (t) is negligibly small is 
greater than 1 - 6 ^ . 

(b) Next, we note that if k m(t) < -1, then we get overmodulation, so that even in the 

absence of noise, the envelope detector output is badly distorted. Therefore, in order to 
avoid overmodulation, we assume that k g is adjusted relative to the message signal m(t) 
such that the probability 

P(A c + A c k g m(t) + n Q (t) < 0) = 6 2 
Then, the probability of the event 

y(t) = A q [1 + k g m(t)] + n c (t) 
for any value of t, is greater than (1 -6^)(1 -62)- 

(c) When 6 1 and 6 2 are both small compared with unity, we find that the probability of 
the event 



y(t) = A [1 + k m (t)] + n (t) 

Co C 

for any value of t, is very close to unity. Then, the output of the envelope detector is 
approximately the same as the corresponding output of a coherent detector. 



Problem 2.52 

The received signal is 
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x(t) = A q cos(2Trf c t) + n(t) 

= A c cos(2Ttf c t) + n c (t) cos(2Tif c t) - n g (t) sin(2Ttf c t) 
= t A c + n (t>] cos(2Trf t) - n (t) sin(2irf t) 



The envelope detector output is therefore 

a(t) = {[A + n (t)] 2 + n 2 (t)} 1/2 
c c s 

For the case when the carrier-to-noise ratio is high, we may approximate this result as 
a(t) = A c + n c (t) 

The term A c represents the useful signal component. The output signal power is thus A^. 
The power spectral densities of n(t) and n (t) are as shown below: 



c 
2W 



V 2 



S (f) 

N x 



2W 



N, 



-WOW 



The output noise power is 2N Q W. 



The output signal -to- noise ratio is therefore 



(SNR) 



0 " 2N Q W 
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Problem 2.53 



(a) From Section 1.12 of the textbook we recall that the envelope r(t) of the narrow -band noise 
n(t) is Rayleigh distributed; that is 



-^exp 



( 2 \ 



G 



N 



2a 



2 2 

where a N is the variance of the noise n(t). For an AM system, the variance a N is 2WN 0 . 

Therefore, the probability of the event that the envelope R of the narrow-band noise n(t) is large 
compared to the carrier amplitude A c is defined by 



P(R^A C ) = f A f R (r)dr 



-i 



a„ 2WN 



■exp 



r 2 \ 
r 



0 



AWN 



dr 



= exp 



v 



AWN, 



(1) 



Define the carrier to noise ratio as 



P = 



average earner power 



average noise power in bandwidth of the modulated message signal 



(2) 



Since the bandwidth of the AM signal is 2W, the average noise power in this bandwidth is 2WNq. 

2 

The average power of the carrier is A /2 . The carrier-to-noise ratio is therefore 



A 



AWN, 



(3) 



o 



(b) We may now use this definition to rewrite Eq. (1) in the compact form 
P(R>A C ) = exp(-p) 



(4) 



Solving P(R > A c ) = 0.5 for p, we get 
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p = log2 = 0.69 

Similarly, for P(R > A c ) = 0.01, we get 
p = logl00 = 4.6 

Thus with a carrier-to-noise ratio 101og 10 0.69 = -1.6 dB, the envelope detector is expected to be 
well into the threshold region, whereas with a carrier-to-noise ratio 101og 10 4.6 = 6.6 dB, the 
detector is expected to be operating satisfactorily. We ordinarily need a signal-to-noise ratio 
considerably greater than 6.6 dB for satisfactory intelligibility, and therefore threshold effects are 
seldom of great importance in AM receivers using envelope detection. 



Problem 2.54 

(a) Following a procedure similar to that described for the case of an FM system, we find 
that the input of the phase detector is 

v(t) = k cos[2rf t + e(t)] 
c c 

wher e 

n Q U) 

6(t) = k p m(t) + -f 

with n^Ct) denoting the quadrature noise component. The output of the phase discriminator 
is therefore, 

y(t) = k m(t) + w — 



P A , 



The message signal component of y(t) is equal to k m(t). Hence, the average output 

2 P 
1 power is k p p, where P is the message signal power. 

With the post detection low-pass filter following the phase detector restricted to 
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the message bandwidth W, we find that the average output noise power is 2WN Q /A c . 
Hence, the output signal-to-noise ratio of the PM system is 

k 2 P A 2 

(SNR) 0 = 

(b) The channel signal-to-noise ratio of the PM system is the same as that of the corres- 
ponding FM system. That is, 



(SNR), 



2WN, 



The figure of merit of the PM system is therefore equal to kp P. 
For the case of sinusoidal modulation, we have 



m(t) = A cos(2rft) 
m m 



Hence , 



m 



1 2 

The corresponding value of the figure of merit for a PM system is thus equal to 3 p » 
where g p = k p A^. On the other hand, the figure of merit for an FM system with sinusoidal 
modulation is equal to |- 3 2 . We see therefore that for a specified phase deviation, the 
FM system is 3 times as good as the PM system. 

Problem 2.55 

(a) The power spectral densities of the original message signal, and the signal and noise 
components at the frequency discriminator output (for positive frequencies) are 
illustrated below: 



of- m essaje 




f (kHz) 



c -=>n*-poA«,\4- exfc 

Out puJ- 




f (kHz) 
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f (kHz) 

0 48 
(b) Each SSB modulated wave contains only the lower sideband. Let and kf Q denote the 

amplitude and frequency of the carrier used to generate the kth modulated wave, where f Q = 

M kHz, and k = 1, 2, 12. Then, we find that the kth modulated wave occupies the 

frequency interval (k - 1)f Q < |f| < kf Q . We may define this modulated wave by 

A A 
s k (t) = ^ m(t) cos(2nkf 0 t) + m(t) sin(2irkfgt) 

where m(t) is the original message signal, and m(t) is its Hilbert transform. Therefore, 

2 

the average power of Sj^t) is P/4, where P is the mean power of m(t). 

Vfe may express the output signal-to-noise ratio 
for the kth SSB modulated wave as follows: 

3A 2 k^A 2 P/4) 

(SNR) = 0 rr 

2N Q [k :S fJ - (k - IVfp 

2 2 2 

& hi k* p 

c k f 



8N 0 f^(3k 2 - 3k + 1) 



where A q is the carrier amplitude of the FM wave. For equal signal-to-noise ratios, we 

must therefore choose the A^ so as to satisfy the condition 

— p = constant for k = 1, 2, 12. 

3k - 3k + 1 
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Problem 2.56 



The envelope r(t) and phase \j/(t) of the narrow-band noise n(t) are defined by 



r(t) = j/nfft) + n|(t) 
-1 n Q(t) 



\j/(t) = tan 



\ J 



For a positive-going click to occur, we therefore require the following: 



nj(t) - - A c 

iiQ(t) has a small positive value 



A tan 1 
dt 



( \ 
n Q (t) 



n : (t) 



> 0 



Correspondingly, for a negative-going click to occur, we require 

n x (t) - A c 

riQ(t) has a small negative value 



± tan" 1 
dt 



n Q (t)' 



v J 



< 0 
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Problem 7 S7 



c 

■w- 



oufc 



* - 0 



Let H(f) be V .(f)/V. ( f ) , or the transfer function of the filter, 
o Uw x n 

frequencies, the capacitor behaves as an open circuit. Then, 
R R 



At low 



H(f) » 



r + R 



Thus, the low frequencies of the input are frequency-modulated. At high frequencies , the 
capacitor behaves as a short circuit in relation to the resistor. Then, 



H(f) a 



R + 



^ — * j2ufCR , 



j2irfC 



and 



v out (t) = RC dt V in (t) 

Frequency modulating the derivative of a waveform is equivalent to phase modulating the 
waveform. Thus, the high frequencies of the input are phase modulated. 

Problem 7 ^ 

(a) For the average power of the emphasized signal to be the same as the average power of 
the original message signal, we must choose the transfer function H (f) of the pre- 
emphasis filter so as to satisfy the relation pe 



/ S M (f) d f = / |H pe r S M (f) d f 



With 



s M (f) O 



1 + (f/f Q )2 



0 , 



-W < f < W 



elsewhere . 



H (f) = k(1 + 
pe f Q 



we have 

W 

/ 



df 



-W 1 + (f/f Q ) 



2 W 
= \i f df 

-W 
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Solving for k, we get 



(b) The improvement in output signal-to-noise ratio obtained by using pre-emphasis in the 
transmitter and de-emphasis in the receiver is defined by the ratio 

2W 3 



W 

3/ f 2 lH. (f)| 2 df 
-W de 



2W 3 



W f 2 rt f 
3/ f df 



-W k 2 1 + (f/f Q ) 2 



k 2 (W/f ) 3 

=f (2) 



3C(W/f Q ) - tan "(W/f 0 )] 
Substituting Eq. (1) in (2), we get 
(W/f n ) 2 tan" 1 (W/f.) 

D= ; — (3) 

3C(W/f 0 ) - tan '(W/f 0 )] 

This result applies to the case when the rms bandwidth of the FM system is maintained 
the same with or without pre-emphasis. When, however, there is no such constraint, we 
find from Example 4 of Chapter 6 that the corresponding value of D is 

(W/f n ) 3 

D = 0 (4) 

3C(w/f 0 ) - tan~'(W/f 0 )] 

In the diagram below, we have plotted the improvement D (expressed in decibels) versus the 
ratio W/f Q for the two cases; when there is a transmission bandwidth constraint and when 
there is no such constraint: 
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3a. 



10 * 



Problem 2.59 




31.6 



100 



0 



1 3.16 10 

In a PM system, the power spectral density of the noise at the phase discriminator 
output (in the absence of pre-emphasis and de-emphasis) is approximately constant. 
Therefore, the improvement in output signal-to-noise ratio obtained by using pre-emphasis 
in the transmitter and de-emphasis in the receiver of a PM system is given by 

W 

/ df 
0 



D = 



W 

/ |H. (f)| 2 df 
o de 



With the transfer function H de (f) of the de-emphasis filter defined by 

H de (f) = 1 + (jf/f 0 ) ' 
we find that the corresponding value of D is 
W 



D = 



W 



df 



o 1 + (f/f 0 r 



w/f o 

tan" 1 (W/f 0 ) 

For the case when W = 15 kHz, f Q = 2.1 kHz, we find that D = 5, or 7 dB. The 
corresponding value of the improvement ratio D for an FM system is equal to 13 dB (see 
Example 4 of Chapter Therefore, the improvement obtained by using pre-emphasis and 

de-emphasis in a PM system is smaller by an amount equal to 6 dB. 
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Problem 2.60 



Matlab codes 

'/, Amplitude demodulation 
'/.problem 2.60, CS: Haykin 
'/. Mathini Sellathurai 

clear all 

Ac=l; 

mue=0.5; 

fc=20000; 

fm=1000; 

ts=le-5 ; 

'/. message signal 
t=[0:250]*le-5; 
m=sin(2*pi*fm. *t) ; 
plot(t, m) 
xlabeK'time (s) ') 
ylabel ( ' Amplitude ' ) 
pause 

'/, amplitude modulated signal 
u=AM_mod(mue,m,ts ,f c) ; 
plot(t.u) 

xlabeK'time (s)') 
ylabel ( ' Amplitude ' ) 
pause 

'/. demodulated signal 

[tl, deml]=AM_demod(mue,u,ts,f c 

plot(tl*ts, deml) 

xlabel('time (s)') 

ylabel ( ' Amplitude ' ) 



axis([0 2.5e-3 0 2]) 



function u=AM_mod(mue ,m,ts ,f c) 

'/, Amplitude modulation 

'/.used in problem 2.60, CS: Haykin 

'/, Mathini Sellathurai 

*/. 

t= [0 : length(m) - 1] *ts ; 
c=cos(2*pi*fc.*t) ; 
m_n=m/max(abs(m) ) ; 
u=(l+mue*m_n) .*c; 
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function [t, env]=AM_demod(mue,m,ts,f c) 

'/. Amplitude demodulation 

'/.used in problem 2.60, CS: Haykin 

'/. Mathini Sellathurai 

'/. 

fs=l/ts; 

f sof c=round(f s/f c) ; 
n2=length(m) ; 

v=zeros(l, round (n2/f sof c) ) ; */, initializing the envelope 
R_L=1000; '/. load 
C=0.01e-6; '/. capacitor 

'/.demodulate the envelope 

1=0; v(l)=m(l); 

for k=i :f sof c:n2-f sof c 

1=1+2; 

v(l)=m(k)*exp(-ts/(R_L*C)/f sofc) ; '/. discharging 

v(l+l)=m(k+fsofc) ; '/.charging 

end 

'/, envelope 

t =0:fsofc/2: (length(v)-l)*f sofc/2; 
env=v; 
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Answer to Problem 2.60 




time (s) 

Figure I ; Message si 
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.5 



1 1.5 
time (s) 



Figure *j[ : Amplitude modulated signal 




Figure ^ : Demodulated signal 
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Problem 2.61 



Matlab codes 

'/. Problem 2.61 CS: Haykin 

'/, phase lock loop and cycle slipping 

*/. M. Sellathurai 

'/, time interval 
t0=0;tf=25; 

'/, frequency step =0 . 125 Hz 

delf =0.125; 

u0=[0 -delf*2*pi] ; 

[t,u]=ode23('lin\ [tO tf],u0); plot (t ,u( : , 2)/2/pi+delf ) ; 
xlabeK'Time (s)') 
ylabel('f_i (t), (Hz)') 
pause 

'/. frequency step =0.51 Hz 
delf =0.5; 

u0=[0 -delf*pi*2] ' ; 

[t,u]=ode23('lin',[t0 tf],u0); plot (t ,u( : , 2)/2/pi+delf ) ; 
xlabeK'Time (s)'); 
ylabel('f_i (t), (Hz)'); 
pause; 

'/. frequency step =7/12 Hz 

delf =7/12; 

u0=[0 -delf*pi*2] ' ; 

[t,u]=ode23('lin' , [tO tf],u0); plot(t,u( : , 2)/2/pi+delf ) ; 
xlabeK'Time (s)'); 
ylabel('f_i (t), (Hz)'); 
pause ; 

'/. frequency step =2/3 Hz 
delf =2/3; 

u0=[0 -delf *pi*2] ' ; 

[t,u]=ode23('lin' , [tO tf],u0); plot(t ,u( : ,2)/2/pi+delf ) ; 
xlabeK'Time (s)'); 
ylabel('f_i (t), (Hz)'); 
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function uprim =lin(t,u) 

'/. used in Problem 2.61, CS: Haykin 

'/. PLL 

'/. Transfer function (i+as)/(l+bs) , 

*/. gain K=50/2/pi, 

'/, natural frequency 1/2/pi 

'/. damping 0.707 

'/, Mathini Sellathurai 

uprim(l)=u(2) ; 

uprim(2)=-(l/50+1.2883*cos(u(l)))*u(2)-sin(u(l)); 
uprim=uprim' ; 
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Answer to Problem 2.61 




Time (s) 

Figure I : Variation in the instantaneous frequency of the PLL's voltage con- 
trolled oscillator for varying frequency step A f . (a) A f = 0.125 Hz 



140 



0.6 - 




Time (s) 

Figurt,?; (b) A f = 0.5 Hz 




Time (s) 

Figures; (b) A f = 7/12 Hz 
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Time (s) 

Figure 4; (b) A f = 2/3 Hz 
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CHAPTER 3 



Pulse Modulation 

Problem 3.1 

Let 2W denote the bandwidth of a narrowband signal with carrier frequency f c . The in-phase and 
quadrature components of this signal are both low-pass signals with a common bandwidth of W. 
According to the sampling theorem, there is no information loss if the in-phase and quadrature 
components are sampled at a rate higher than 2W. For the problem at hand, we have 

f c = 100 kHz 
2W= 10 kHz 

Hence, W - 5 kHz, and the minimum rate at which it is permissible to sample the in-phase and 
quadrature components is 10 kHz. 

From the sampling theorem, we also know that a physical waveform can be represented over the 
interval -<» < t < °° by 

oo 

g(t) = £ aj n (t) (1) 

n=-°° 

where {<\> n (t)} is a set of orthogonal functions defined as 
d> (0 = — — — liii 

where n is an integer and f s is the sampling frequency. If g(t) is a low-pass signal band-limited to 
W Hz, and f s > 2W, then the coefficient a n can be shown to equal g(n/f s ). That is, for/ s > 2W, the 
orthogonal coefficients are simply the values of the waveform that are obtained when the 
waveform is sampled every \lf s second. 

As already mentioned, the narrowband signal is two-dimensional, consisting of in-phase and 
quadrature components. In light of Eq. (1), we may represent them as follows, respectively: 

CO 
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Hence, given the in-phase samples SiyyJ an d quadrature samples Sq^j > we ma y 
the narrowband signal g(t) as follows: 

g(0 = gi(t)cos(2nf c t) - g Q (t)sin(2nf c t) 



reconstruct 



= E ^/^Jcos(27t/ c r)-^-Jsin(27t/ c 0j^(0 



where f c = 100 kHz and/ s > 10 kHz, and where the same set of orthonormal basis functions is 
used for reconstructing both the in-phase and quadrature components. 
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Problem 3.2 



(a) Consider a periodic train c(f) of rectangular pulses, each of duration T. The Fourier series 
expansion of c(t) (assuming that a pulse of the train is centered on the origin) is given by 

oo 

c(t) = 52 f s sinc(nf s T)exp(j27tnf s t) 

n=-°<> 

where f g is the repetition frequency, and the amplitude of a rectangular pulse is assumed to be 1/T 
(i.e., each pulse has unit area). The assumption that f g T»l means that the spectral lines (i.e., 
harmonics) of the periodic pulse train c(t) are well separated from each other. 

Multiplying a message signal g(t) by c(t) yields 

s(t) = c(t)g(t) 

= £ f s sinc(nf B T)g(t) exp(j27tnf s (t) (1) 

n = -oo 

Taking the Fourier transform of both sides of Eq. (1) and using the frequency-shifting property of 
the Fourier transform: 

oo 

S(0 = E f s sinc(nf s T) G(f-nf s ) (2) 

n = -oo 

where G(f) = F[g(t)]. Thus, the spectrum S(f) consists of frequency-shifted replicas of the original 
spectrum G(f), with the nthreplica being scaled in amplitude by the factor f sinc(nf T). 
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(b) In accordance with the sampling theorem, let it be assumed that 
• The signal g(t) is band-limited with 



G(f) = 0 for -W < f < W 



0 The sampling frequency f g is defined by 



f s >2W 

Then, the different frequency-shifted replicas of GKf) involved in the construction of S(f) will not 
overlap. Under the conditions described herein, the original spectrum G(f), and therefore the signal 
g(t), can be recovered exactly (except for a trivial amplitude scaling) by passing s(t) through a low- 
pass filter of bandwidth W. 



Problem 3.3 

(a) g(t) = sinc(200t>; 

This sine pulse corresponds to a bandwidth W = 100 Hz. Hence, the Nyquist rate is 200 Hz, 
and the Nyquist interval is 1/200 seconds. 

(b) g(t) = sinc 2 (200t);; 

This signal may be viewed as the product of the sine pulse sinc(200t). with itself. 
Since multiplication in the time domain corresponds to convolution in the frequency 
domain, we find that the signal g(t) has a bandwidth equal to twice that of the sine 
pulse sin(200t)i, that is, 200 Hz. The Nyquist rate of g(t) is therefore 400 Hz, and the 
Nyquist interval is 1/400 seconds. 

(c) g(t) = sinc(200t).'. + sine 2 (200t).; 

The bandwidth of g(t) is determined by the highest frequency component of sinc(200t) or 
sine 2 (200t):, whichever one is the largest. With the bandwidth (i.e., highest frequency 
component of) the sine pulse sine (200 t) equal to 100 Hz and that of the squared sine 
pulse sinc 2 (200t):; equal to 200 Hz, it follows that the bandwidth of g(t") is 200 Hz. 
Correspondingly, the Nyquist rate of g(t) is 400 Hz, and its Nyquist interval is 1/400 
seconds . 
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Problem 3.4 

(a) The PAM wave is 

00 

s(t) = Z [1 + ym' CnT )]g(t-nT ), 

— 5 S 

n=-°° 

where g(t) is the pulse shape, and m' (t) = m(t)/A = cos(2uf t) . The PAM wave is 

mm 

equivalent to the convolution of the instantaneously sampled [1 + Mm' (t)] and the pulse 
shape g(t) : 

00 

s(t) = { Z [1+um'(nT )] 6(t-nT )} ^ g(t) 
n=-» S S 

00 

= {[1+yra'(t)] Z 6(t-nT )} ^ g(t) 

S 

n=-°° 

The spectrum of the PAM wave is, 



S(f) = {[6(f) + M M'(f)] f- I 5(f - f-)} G (f) 



s m=-°° 



= l-G(f) Z [6(f _ 5_) + yM , (f _ 
s m=-~ x s T s 

For a rectangular pulse g(t) of duration T=0.il5s, and with AT = 1 , we have: 
G(f) = AT sinc(fT) 

= sinc(0.45f) 

For m«(t) = cos(2»fmt), and with f m = 0.25 Hz, we have: 
M»(f) = \ [6(f_ 0 .25) + 6(f+0.25)] 



Fbr T g = 1s, the ideally sampled spectrum is S.(f) = z 

o 



[6(f-m) + pM'(f-m)]. 



S 



MA 



-US -10 -0.1$ -0.25 q 

The actual sampled spectrum is 



^M/x ^ \ AA 

\ 

"0.2S 



i 1 



m=-°° 



JL 



0.75 1.0 



y^A 

1_ 



■as 



147 



S(f) = Z sinc(0.45f)[6(f-m) + uM'(f-m)] 

m =-°° Scf) 



— — < — /' -y- a, 

2 t 1 t L_ 



t 



(b) The ideal reconstruction filter would retain the centre 3 delta functions of S(f) or: 



z 



i 2. 



-Qis 0 O.i s 1 



With no aperture effect, the two outer delta functions would have amplitude ^. Aperture 
effect distorts the reconstructed signal by attenuating the high frequency portion of the 
message signal. 

Problem 3.5 

The spectrum of the flat-top pulses is given by 
H(f) = rsinc(/T)exp(-77c/T) 

= 10" 4 sine ( 10" 4 /) exp (-jnf 10" 4 ) 
Let s(t) denote the sequence of flat-top pulses: 

oo 

s(t) = £ m(nT s )h(t-nT s ) 

n=-°° 

The spectrumi(/) = F[s(t)] is as follows: 



S(f) = f^MV-kfsWV) 

k=-°° 



= /,#(/) 

k=-°° 

The magnitude spectrum |5(/)| is thus as shown in Fig. lc. 
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S(f) 




1/T = 10,000Hz 
f s = 1,000Hz 
W = 400Hz 

Figure 1 
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Problem 3.6 



At f= 1 12T which corresponds to the highest frequency component of the message signal for a 
sampling rate equal to the Nyquist rate, we find from Eq. (W9) that the amplitude response 



sine (0.5 T/T s ) 




0.2 



0.4 

Duty cycle T/T 5 



0.6 



Figure I 



of the equalizer normalized to that at zero frequency is equal to 

' = W2XTO 

sinc(0.5 T/T s ) sm[(n/2)( T/T s )] 

where the ratio T/T s is equal to the duty cycle of the sampling pulses. In Fig. I , this result is 
plotted as a function of T/T s . Ideally, it should be equal to one for all values of T/T s . For a duty 
cycle of 10 percent, it is equal to 1.0041. It follows therefore that for duty cycles of less than 10 
percent, the aperture effect becomes negligible, and the need for equalization may be omitted 
altogether. 



150 



Problem 3.7 



Consider the full-load test tone A oostz^t) . Denoting the icth sample a.piitude of 
this signal by A k , „ find tnat the transmitted pulse is A k g(t), where g(t) is defined by 
the spectrum: 




otherwise 



The mean value of the transmitted signal power is 

1 LT L 
P = EUim -r^r- f S [ Z A. g(t)] 2 dt} 

L+- -LT k= -L K 

s 



1 LT L L 

= E[lim ^-=- f 5 Z Z A A g 2 (t)dt] 

L-»~ <s -LT k=-L n=-L K n 
s 



1 L L LT o 

= lim ^p- Z Z E[A.A ] I S g 2 (t)dt 

L-~ ^ L1 s k =-L n=-L k " -LT 

s 



where T g is the sampling period. However, 

,2 



^— » k = n 



E[A A ] = 
k n J 



0, otherwsie 
Therefore, 

2 00 

P = 2f- / g 2 (t)dt 
s -°° 

Using Rayleigh's energy theorem, we may write 



/ g 2 (t)dt = / |G(f)| 2 df 



B 2 

/ (off-) df 
-B T 2B T 

_L 151 
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Therefore, 



The average signal power at the receiver output is A /2. Hence, the output signal- 
to-noise ratio is given by 



(SNR) o = 

A 2 



2B T N 0 



2T P 
s 



N 0 



By choosing B =1/2T , we get 



(SNR) 0 = 



T s' 
P 



0 " B T N 0 



This shows that PAM and baseband signal transmission have the same signal-to-noise ratio 
for the same average transmitted power, with additive white Gaussian noise, and assuming 
the use of the minimum transmission bandwidth possible. 



Problem 3.8 

(a) The sampling interval is T = 125 y s. There are 24 channels and 1 sync pulse, so the 
time alloted to each channel ii T Q = T g /25 = 5 y s. The pulse duration is 1 M s , so the 
time between pulses is 4 Vs. 

(b) If sampled at the nyquist rate, 6.8 kHz, then T s = 147 Vs, T, = 6.68 vs, and the time 
between pulses is 5.68 vs. 



Problem 3.9 

(a) The bandwidth required for each single sideband channel is 10 kHz. The total 
bandwidth for 12 channels is 120 kHz. 

(b) The rv/yquist rate for each signal is 20 kHz. For 12 TDM signals, the total data rate 
is 240 kHz. By using a sine pulse whose amplitude varies in accordance with the 
modulation, and with zero crossings at multiples of (1/240) ms, we need a minimum 
bandwidth of 120 kHz. 
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Problem 3.10 



(a) The Nyquist rate for s^t) and s 2 (t) is 160 Hz. Therefore, must be greater than 
160, and the maximum R is 3. 2 

(b) With A = 3, we may use the following signal format to multiplex the signals s^t) and 
s 2 (t) into a new signal, and then multiplex s^(t) and s^(t) and s,_(t) including markers 
for synchronization: 







' s 




— &i 





















III 








.j — 


till 


1 I 



a zero sa.tr>fi&.S 



Based on this signal format, we may develop the following multiplexing system: 
2400 Hz 



< 


1 tt— 


M<xrfe«-r 






^-5 


1 
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Problem 3.11 



In general, a line code can be represented as 



N 



s(t) = £ a n g{t-nT b ) 



n=-N 



Let g(t) ^ G(f) . We may then define the Fourier transform of s(t) as 



N 



S(f) = X a n G(f)e 



n=-N 



N 



= G(f) £ a n e 



n=-N 



where co = 2%f. The power spectral density of s(t) is 



S s (f) = lim 



f\G(f)\ 2 E 



N 



-jmnT, 



a e 

n 



n=-N 



= W)\ 2 lim 



( N N 



1 V V ET 1 j{m - n)mT b 

tZj L E ^ a n a m\ e 



V n=-N m=-N 



where T is the duration of the binary data sequence, and E denotes the statistical expectation 
operator. Define the autocorrelation of the binary data sequence as 

R(k) = E[a n a n + k ] 

By letting m = n + k and T = (2N + l)T b , we may write 



S s (f) = \G(ft lim 



1 



n=N k=N-n 



(2N+l)T b 



n=-N k=-N-n 



Replacing the outer sum over the index n by 2N+1, we get 



S s (f) = ^ 2 hm 

l b N — > ■» 



2, R ( k ) e 



2N+1 



k=-N-n 
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= \G(f)\ 2 



jk(»T b 



(1) 



k=-°° 



where 



R(k) = E[a n a n + k ] = J J (a n a n + k )p i 



(2) 



where p t is the probability of getting the product (a n a n+k ) i and there are / possible values for the 
a n a n+k product. G(f) is the spectrum of the pulse-shaping signal for representing a digital symbol. 
Eqs. (1) and (2) provide the basis for evaluating the spectra of the specified line codes. 

(a) Unipolar NRZ signaling 

For rectangular NRZ pulse shapes, the Fourier- transform pair is 



For unipolar NRZ signaling, the possible levels for a's are +A and 0. For equiprobable 
symbols, we have the following autocorrelation values: 




1(0) 




4 



R(k) 



J,( a n a n + k)iPi 




= for |£|>0 
4 



Thus 



R(k) = . 



A 2 /2 for k = 0 
A 2 /4 for 



(3) 
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Therefore, the power spectral density for unipolar NRZ signals, using formulas (1) and (3), is 



„ m _ \AT b m^{fT b )\ 

1 b 



1 1 _ J2nkfT b 

4 + 4 5> 



7 

AT 



-sinc 2 (/Tfc) 



1+ 5> 

k=-° 



j2nkfT b 



But, 



CO 

j2nkfT b i j 



k=- 



T 



where 8(f) is a delta function in the frequency domain. Hence, 



A2 T b 2 



oo 

i4 2«f/-f 



We also note that smc(fT b ) = 0 at / = — , n * 0 ; we thus get 



2 

AT 



b ■ 2, 



s s(f) = -r^sinc (/r.) 



T u . 



(b) Polar Non-return-to-zero Signaling 

For polar NRZ signaling, the possible values for a's are +A and -A. Assuming equiprobable 
symbols, we have 



i=l 



a 2 (-Ay 

2 2 



= A 



For k * 0 , we have 
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R(k) = ^(a n a n + k ). Pi 



(=1 



= A i 2 { ~ A){A) i 2 ( ~ A)(A) i ( ~ A) ^ 
4 4 44 



= 0 



Thus, 



R(k) = 



A for k = 0 
0 for 



(4) 



The power spectral density for this case, using formulas (1) and (4), is 



S(f) = A 2 T b smc 2 (fT b ) 



(c) Return-to-zero Signaling 



The pulse shape used for return-to-zero signaling is given by g( t j . We therefore have 



G(f) = -fsmc(fT b /2) 

The autocorrelation for this case is the same as that for unipolar NRZ signaling. Therefore, the 
power spectral density of RZ signals is 



^Tb 2 



(d) Bipolar Signals 



The permitted values of level a for bipolar signals are +A, -A, and 0, where binary symbol 1 is 
represented alternately by +A and -A, and binary 0 is represented by level zero. We thus have 
the following autocorrelation function values: 
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R(0) = 



A 



AT 
' 4 



For k > 1 , 



Thus, 



A 



A 



for it = 0 



for = 1 



0 for \k\ > 1 



(5) 



The pulse duration for this case is equal to 7y2. Hence, 



G(f) = ysmc — 



(6) 



Using Equations (1), (5) and (6), the power spectral density of bipolar signals is 



S s (f) = 



T b . (fT b 
—sine — 
2 2 



'A 2 A 2 J® T b A 2 -j<»T b - 



A T b . 2(f T b\[, 1, i mT b ~^ T b 
— - — sine — — 



l--(e +e ") 



A T b . 2(f T b) r . /0 

— g— smc I — J[l-cos(27i/r fc )] 
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A T b 2(fT b \ 2 
= — g— sine ^— Jsin (nfT b ) 



(e) Manchester Code 

The permitted values of a's in the Manchester code are +A and -A. Hence, 



R(0) = l -A 2 + l -(-A) 2 + l -(-A) 2 + 1 -(A 2 ) 



= A 



For k * 0 , 



V/ n A (-A)(A) A(-A) {-AY 



i=\ 



= 0 



Thus, 



R(k) = 



A for k = 0 
0 for 



The pulse shape of Manchester signaling is given by 



-t + T b /4\ ft-T h /A\ 
g{t) = rectj „ t l-rect' * 1 



V2 



r,/2 ; 



The pulse spectrum is therefore 



T b . (fT b \ j°>T b /4 T b . //Ta -jnT b M 



G(f) = ysincl — \e " -^sinc[^je 



= 7 T,smc^— Jsin^— j 
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Therefore, the power spectral density of Manchester NRZ has the form 



c/f s ,2„ . 2(f T b\ ■ l( n J' l 'b\ 

S s (f) = A 7^ sine [—J sm J 



Problem 3.12 

Power spectral density of a binary data stream will not be affected by the use of differential 
encoding. The reason for this statement is that differential encoding uses the same pulse shaping 
functions as ordinary encoding methods. If the number of bits is high, then the probability of a 
symbol one and symbol zero are the same for both cases. 

Problem 3.13 

(a) 




(b) g(t) = 



cos fj, —<t<- 



0, 



otherwise 



Equivalently, we may write 



g(t) = cos^^Arect^ 



where rect(?) is a rectangular function of unit amplitude and unit duration. The Fourier 
transform of g(t) is given by 



G(f) = 



AT, 



* sinc(/r fc ) 



where A denotes the pulse amplitude and * denotes convolution in the frequency domain. 
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Using the replication property of the delta function 8(f), we get 



G(f) = 



AT, 



sinc[rJ/-^]) + sinc[r fc (/ + ^ 



T 



Using Eq. (1.52) of the textbook, the power spectral density of the binary data stream is 



S(f) = 



\G(f)\ x 



7 

AT, 



sim X T {f-fj) +sinc 2 (r,(7-V 



+ 2siiic[r^/-^sincrr// + | 



(1) 



Note that the two spectral components sine (V^ / - ^-Jj and sine [jj[f + jrjj overlap in 

the frequency interval -(1/T b ) <f< (l/T b ), hence the presence of cross-product terms in Eq. 
(1). 

Figure 1 plots the normalized power spectral density 5 , (/)/(A 2 T^/4) versus the normalized 
frequency jT b . The interesting point to note in this figure is the significant reduction in the 
power spectrum of the pulse-shaped data stream x(t) in the interval - l/T b <f< 1/T b . 

(c) The power spectral density of the standard form of polar NRZ signaling is 



S(f) = A 2 T b smc 2 (jT b ) 



(2) 



Comparing this expression with that of Eq. (1), we observe the following differences: 





Polar NRZ signals using 


Polar NRZ signals using 




cosine pulses 


rectangular pulses 




0 


A 2 T b 


f= &Tb 


A 2 7y4 


0 
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Problem 3.15(a) 



-1—1-4- 






— \— 


Iff- 

44-^4 


L. ._ 




Problem 3.16 



The minimum number of bits per sample is 7 for a signal-to-quantization noise ratio of 40 dB. 
Hence, 



f The number of samples^] - gQOO x 10 



V in a duration of 10s J o.-m 4 i 

= 8 x 10 samples 



The minimum storage is therefore 

= 7 x 8 x 10 4 

= 5.6x 10 5 
= 560 kbits 
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Problem 3.17 



Suppose that baseband signal m(t) is modeled as the sample function of a Gaussian random 
process of zero mean, and that the amplitude range of m(t) at the quantizer input extends from 
-4.^^ to ^Aj^g. We find that samples of the signal m(t) will fall outside the amplitude range SA^g 
with a probability of overload that is less than 1 in 10 4- If we further assume the use of a binary 
code with each code word having a length n, so that the number of quantizing levels is 2", we find 
that the resulting quantizer step size is 



8 = 8Arms (1) 
2* 

Substituting Eq. (1) to the formula for the output signal-to-quantization noise ratio, we get 



(SNR) 0 = JL(2 2 ft) (2) 
0 16 



Expressing the signal-to-noise ratio in decibels: 



101og 0 (SNR) o = 6H - 7.2 (3) 

This formula states that each bit in the code word of a PCM system contributes 6dB to the signal- 
to-noise ratio. It gives a good description of the noise performance of a PCM system, provided that 
the following conditions are satisfied: 

1. The system operates with an average signal power above the error threshold, so that the 
effect of transmission noise is made negligible, and performance is thereby limited 
essentially by quantizing noise alone. 

2. The quantizing error is uniformly distributed. 

3. The quantization is fine enough (say ft > 6) to prevent signal-correlated patterns in the 
quantizing error waveform. 

4. The quantizer is aligned with the amplitude range from 4^ to 4A rms . 

In general, conditions (1) through (3) are true of toll quality voice signals. However, when demands 
on voice quality are not severe, we may use a coarse quantizer corresponding to < 6. In such a 
case, degradation in system performance is reflected not only by a lower signal-to-noise ratio, but 
also by an undesirable presence of signal-dependent patterns in the waveform of quantizing error. 
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Problem 3.18 



(a) Let the message bandwidth be W. Then, sampling the message signal at its Nyquist 
rate, arid using an ft-bit code to represent each sample of the message signal, we find that 
the bit duration is 



T 

T — 1 
l b = o = 2WR 



The bit rate is 

Y- = 2WR 
b 

The maximum value of message bandwidth is therefore 
w _ 50 x 10 6 



max 2x7 

= 3.57 x 10 6 Hz 

(b) The output signal-to-quantizing noise ratio is given by (see Example 2): 
10 log 10 (SNR) 0 = 1.8 + 6R 

= 1.8 + 6x7 
= 43.8 dB 

Problem 3.19 

Let a signal amplitude lying in the range 
*i " \ 6 i < * < x t + 1 6. 

be represented by the quantized amplitude x., . The instantaneous square value of the error 
2 

is (x-Xj^) . Let the probability density function of the input signal be f x (x). If the 
step size 6^^ is small in relation to the input signal excursion, then f x (x) varies little 
within the quantum step and may be approximated by ^^^^)» Then, the mean-square value of 
the error due to signals falling within this quantum is 

x. + 1 6. 
E[qJ] = / 1 (x- Xi ) 2 f x (x)dx 
X i " 2 6 i 
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X i + 2 6 i 



X i ~ 2 6 i 



(x-x^ f x (x i )dx 



x i + 2 6 i 



(x-x^ dx 



X i " 2 6 i 



1 * 

? 1 2 
= f Y (x.) / x dx 

1 

= TI 6 iW < 1 > 

The probability that the input signal amplitude lies within the ith interval is 
X i + 2 6 i X i + 2 6 i 

Pi = 1 1 , f x (x)dx " f x (x i> ' 1 , dx = f X (x i )6 i (2) 

X i " 2 6 i X i " 2 6 i 

Therefore, eliminating f^Cxj) between Eqs. (1) and (2), we get 

: ECQ i ] =4 Pi 6 i 

The total mean-square value of the quantizing error is the sum of that contributed by each 
of the several quanta. Hence, 

I E[Q f ] = 4 I Pi 6 f 
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Problem 3.20 




no 



Problem 3.21 



The quantizer has the following input-output curve: 



oers 8 
■1 -5 .-3 .\ ■ 




— / I 1 1 1 II 


r~ , , , , , — 

OOOO I—* 


0 * x 3 < 5 4 1 i^ui. 


-8 



At the sampling instants we have: 



t m(t) code 

-3/8 -3/2 0011 

-1/8 -3/2 0011 

+1/8 3/? 1100 

+3/8 3/2 1100 



And the coded waveform is (assuming on-off signaling): 



T\me (.Seconds} 



Problem 3.?.?. 

The transmitted code words are: 



t/T b 


code 


1 


001 


2 


010 


3 


011 


4 


100 


5 


101 


6 


110 
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The sampled analog signal is 



3% 



'*5 



'ST, 



Problem 3.23 

(a) The probability p 1 of any binary symbol being inverted by transmission through the 

system is usually quite small, so that the probability of error after n regenerations in 

the system is very nearly equal to n p,. For very large n, the probability of more than 

one inversion must be taken into account. Let p R denote the probability that a binary 

symbol is in error after transmission through the complete system. Then, p R is also the 

probability of an odd number of errors, since an even number of errors restores the 

original value. Counting zero as an even number, the probability of an even number of 

errors is 1-p . Hence 
n 

p n + 1 = Pn (1 -Pl )+(1 -Pn )p 1 
= (1 -*1>V p 1 

This is a linear difference equation of the first order. Its solution is 
P n = \ [1-0-2?,)"] 

(b) If p.| is very small and n is not too large, then 
(1-2^)" * 1-2p.,n 

and 

P n * |[1-(1-2 Pl n)] 
= P-,n 
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Problem 3.24 - Regenerative repeater for PCM 

Three basic functions are performed by regenerative repeaters: equalization, timing and decision- 
making. 

Equalization : The equalizer shapes the incoming pulses so as to compensate for the effects of 
amplitude and phase distortion produced by the imperfect transmission characteristics of the 
channel. 



Timing : The timing circuitry provides a periodic pulse train, derived from the received pulses, for 
sampling the equalized pulses at the instants of time where the signal-to-noise ratio is maximum. 

Decision-making : The extracted samples are compared to a predetermined threshold to make 
decisions. In each bit interval, a decision is made whether the received symbol is 1 or 0 on the 
basis of whether the threshold is exceeded or not. 



Problem 3.25 



m(t) = Atanh(P?) 



To avoid slope overload, we require 



> max 



dm(t) 



dt 



(1) 



^ = A P sech 2 (P 0 (2) 
Hence, using Eq. (2) in (1): 

A>max(Apsech 2 ((3r)) x T s (3) 
Since sech(fk) = ^ 



cosh(P?) 



e +e 



it follows that the maximum value of sech(P0 is 1, which occurs at time t - 0. Hence, from Eq. (3) 
we find that A>A$T S . 
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Problem 3.7.6 

The modulating wave is 

m(t) = k cos(27rf t) 
m m 

The slope of m(t) is 

= — 2irf A sin(27rft) 
at mm m 



The maximum slope of m(t) is equal to 2irf A , 

mm 



The maximum average slope of the approximating signal m.(t) produced by the delta 
modulator is 6/T , where 6 is the step size and T is the sampling period. The limiting 

5 S 

value of A m is therefore given by 



or 



2nf A > f- 

m m T 

s 



A > - * 

m 2Trf T 
m s 



2 

Assuming a load of 1 ohm, the transmitted power is A /2. Therefore, the maximum 

m 

2 2 2 2 

power that may be transmitted without slope-overload distortion is equal to 6 /8tt f T . 

m s 
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Problem 3.27 

fs ~ ^Q/kyquist 
/Nyquist = 6-8 kHz 

f s = 10 x 6.8 x 10 3 = 6.8 x 10 4 Hz 



A . 

— > max 

s 



dm(t) 



dt 



For the sinusoidal signal m{t) - A m sm(2nf m i), we have 



dm(t) _ 



dt 



= 2nf m A m cos(2nf m t) 



Hence, 



dm(t) 



dt 



"ilmax 



or, equivalently, 



A 



mlmax 



Therefore, 



\A. 



A 



mimax r, X 27C X /„ 



0.1 x6.8 x 10 
2nx 10 3 



= 1.08V 
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Problem 3.28 



(a) From the solution to Problem 3.27, we have 



A = - — — or A = 



(1) 



A 

The average signal power = — 



_ If 

2{2nf ri 



With slope overload avoided, the only source of quantization of noise is granular noise. 
Replacing A/2 for PCM with A for delta modulation, we find that the average quantization 

noise power is A 2 /3; for more details, see the solution to part (b) of Problem 3.30. The 
waveform of the reconstruction error (i.e., granular quantization noise) is a pattern of bipolar 
binary pulses characterized by (1) duration = T s = l/f s , and (2) average power = A/3. Hence, 
the autocorrelation function of the quantization noise is triangular in shape with a peak value 
of A z /3 and base 2T S , as shown in Fig. 1: 



Fig. 1 




From random process theory, we recall that 



S Q (f)\ f=0 = f_R Q (t)dT 



which, for the problem at hand, yields 



S Q (0) = 



2 

AT. 



^1 
3/, 



Typically, in delta modulation the sampling rate f s is very large compared to the highest frequency 
component of the original message signal. We may therefore approximate the power spectral 
density of the granular quantization noise as 
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s Q (f) 



A /3f s -W<f<W 
0, otherwise 



where W is the bandwidth of the reconstruction filter at the demodulator output. Hence, the 
average quantization noise power is 



w 

N = j S Q (f)df = 
-w 



2A 2 W 



3/, 



(2) 



Substituting Eq. (2) into (1), we get 



N = 2 



fs J 3/ 



8ti VaV 



3/ 



(b) Correspondingly, output signal-to-noise ratio is 



SNR = 



(Sn 2 f 2 m A 2 W)/3f 3 s 



3/J 



16te V> 



Problem 3.29 
(a) A<— 'f- 

2n fm 
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. ^2x7tx 10 3 x 1 
A> 

50 x 10 



= 0.126 V 



(b) (SNR) 0Ut = -L ^f- 



3 (50xl0 3 ) 3 
16ti 2 10 6 x5xl0 3 



= 475 
In decibels, 

(SNR) out = 101og 10 475 
= 26.8 dB 



Problem 3.30 

(a) For linear delta modulation, the maximum amplitude of a sinusoidal test signal that can be 
used without slope-overload distortion is 

A = A/ - 



a ' X60Xl0 3 3 / 5 = 2X3X10 3 

271 x 1 x 10 



= 0.95V 

(b) (i) Under the pre-filtered condition, it is reasonable to assume that the granular quantization 
noise is uniformly distributed between -A and +A. Hence, the variance of the quantization 
noise is 
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A 

G Q = J 2A q dq 

-A 



1 3 A 

= 6A [ * ] -a 



A 2 



The signal-to-noise ratio under the pre-filtered condition is therefore 



2 

(SNR) A 72 



'prefiltered 2 



A /3 



3A 2 



2A 2 



3 xO.95 2 
2x0.1 2 



= 135 

= 21.3 dB 

(ii)The signal-to-noise ratio under the post-filtered condition is 



M J 2 ? 

iv /postfiltered 167T f W 



_3_ x (60) 3 



16ji 2 (I) 2 x3 



= 1367 
= 31.3 dB 



The filtering gain in signal-to-noise ratio due to the use of a reconstruction filter at the 
demodulator output is therefore 31.3 - 21.3 = 10 dB. 
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Problem 3.31 

Let the sinusoidal signal m(t) = Asmm 0 t, where 0) 0 = 2nf 0 
The autocorrelation of the signal is 

A 2 

R m( X ) = yCOS((0 0 T) 



A 2 

= y 



A 2 f 1 

*»(D = ycos(rcoX — 



A 2 

yCOS(O.l) 



For this problem, we thus have 

R m = [tf B (0)], r m = [RJ1)] 

(a) The optimum solution is given by 



W 0 = R m' r m 



A 2 

ycos(0.1) 

5 = cos(O.l) 

A 



= 0.995 



(b) L = ^(O)-^; 1 ^ 



.2 .2 2 

= y-yCOs(0.1)Xycos(0.1)/(A 2 /2) 
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y(l-COS 2 (0.1)) 



= 0.005A 
Problem 3.32 



1 0.8 0.6 
0.8 1 0.8 
0.6 0.8 1 



T 

= [o.8, 0.6, 0.4] 



-1 



(a) w 0 = R x r x 



1 0.8 0.6 


-1 


0.8 


0.8 1 0.8 




0.6 


0.6 0.8 1 




0.4 



0.875 
0 

-0.125 



(b) J mi n = *,((>) -r^r, 



^(0)-r> 0 



= 1 - [0.8, 0.6, 0.4] 



0.875 
0 

-0.125 



1 - (0.8 x0.875- 0.4 x 0 
1-0.7 + 0.05 



= 0.35 



Problem 3.33 



x L°-8 1. 

T 

r x = [o.8, 0.6] 

(a) w 0 = R^r x 

0.8889 
-0.1111 

(b) Jmin = ^(O)-rX 1 ^ 

= 1-0.6444 
= 0.3556 

which is slightly worse than the result obtained with a linear predictor using three unit 
(i.e., three coefficients). This result is intuitively satisfying. 

Problem 3.34 

Input signal variance = R x (0) 

The normalized autocorrelation of the input signal for a lag of one sample interval is 

R x( l ) 

p ' (1) = ^o) = 0 - 75 

Error variance = R x (0) - R^R' 1 (0)R x (l) 
= R x (0)(l - p 2 x (l)) 
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Processing gain = 

/? x (0)(l-p^(l)) 

_ 1 
l-p'(l) 

= 1 
l-(0.75) 2 

= 2.2857 

Expressing the processing gain in dB, we have 
101og 10 (2.2857) = 3.59 dB 



Problem 3.35 



Processing gain = 



*,(0) 



(a) Three-tap predictor: 
Processing gain = 2.8571 

= 4.56 dB 

(b) Two-tap predictor: 
Processing gain = 2.8715 

= 4.49 dB 

Therefore, the use of a three-tap predictor in the DPCM system results an improvement of 
4.56 - 4.49 = 0.07 dB over the corresponding system using a two-tap predictor. 



Problem 3.36 



(a) For DPCM, we have 101og 10 (SNR) 0 = a + 6n dB 



For PCM, we have 101og 10 (SNR) 0 = 4.77 + 6n - 201og 10 (log(l + 



where n is the number of quantization levels 
SNRof DPCM 
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SNR = a + 6n, where -3 < a < 15 

For n=8, the SNR is in the range of 45 to 63 dBs. 

SNR of PCM 

SNR = 4.77 + 6n - 201og 10 (log(2.56)) 

= 4.77 +48 - 14.8783 
= 38dB 

Therefore, the SNR improvement resulting from the use of DPCM is in the range of 7 to 25 
dB. 

(b) Let us assume that n x bits/sample are used for DPCM and n bits/sample for PCM 
If a = 15 dB, then we have 
15 + 6^ = 6w- 10.0 

d i ,10+15 

Rearranging: (n-n x ) - — - — 

= 4.18 

which, in effect, represents a saving of about 4 bits/sample due to the use of DPCM. 

If, on the other hand, we choose a = -3 dB, we have 

-3 + 6ni = 6n-10 
Rearranging: (n-n x ) = — 

_ 7 

6 

= 1.01 

which represents a saving of about 1 bit/sample due to the use of DPCM. 
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Problem 3.37 



The transmitting prediction filter operates on exact samples of the signal, whereas the receiving 
prediction filter operates on quantized samples. 

Problem 3.38 

Matlab codes 

*/. Problem 3.38, CS: Haykin 
'/,f lat -topped PAM signal 
'/.and magnitude spectrum 
'/. Mathini Sellathurai 

'/.data 

fs=8000; '/, sample frequency 
ts=1.25e-4; '/.1/fs 

pulse_duration=5e-5; '/.pulse duration 
'/. sinusoidal sgnal; 

td=1.25e-5; '/.sampling frequency of signal 

fd=80000; 

t=(0:td: 100*td) ; 

fm=10000; 

s=sin(fm*t) ; 

'/. PAM signal generation 

pam_s=PAM(s,td,ts,pulse_duration) ; 
f igure(l) ;hold on 
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plot(t,s, « — '); 

plot (t(l: length (pam_s)) ,pam_s) ; 

xlabel('time') 

ylabel( 'magnitude' ) 

legend ( ' s ignal ' , ' PAM-s ignal ' ) ; 

'/. Computing magnitude spectrum S(f ) of the signal 
a= ( (abs (f f t (pam_s ) ) . ~2) ) ; 
a=a/max(a) ; 

f =f s*(f s/f d:f s*(f s/f d) : (length(a) )*f s*(f s/fd) ; 

figure(2) 

plot (f, a); 

xlabel ( ' frequency ' ) ; 
ylabel( 'magnitude') 

'/, finding the zeros 
index=f ind(a<le-5) ; 

'/. finding the first zero 

fprintf ('Envelopes goes through zero for the first time = '/.6d\n', min(index)*f s*(fs/fd)) 
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function pam_s=PAM(s,td,ts,pulse_duration) 



'/. Problem 3.38, CS: Haykin 

'/.flat -topped PAM signal 

'/.used in Problem 3.38, CS: Haykin 

'/. Mathini Sellathurai 

potd=pulse_duration/td; 
tsotd=ts/td; 

y=zeros(i,length(s)) ; 
tt=l : (tsotd) : length(s) ; 

for kk=l:length(tt) ; 

y (tt (kk) : tt (kk)+potd-l )=s (tt (kk) ) . *ones ( 1 , potd) ; 
end 

pam_s=y(l:length(s)-potd) ; 
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Answers: 3.38 
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0.8 - 
0.7 - 




Figure 2: Magnitude spectrum of flat-topped PAM signal 




frequency 

x 10 4 



igure .'.3: Zoomed magnitude spectrum of flat-topped PAM signal 



190 



Problem 3.39 



Matlab codes 

'/.problem 3.39, CS: Haykin 
'/.mue-law pCM and uniform quantizing 
'/.Mathini Sellathurai 
clear all 

'/.sinusoidal signal 
t=[0:2*pi/100:2*pi] ; 
a=sin(t) ; 

'/. input signal to noise ratio in db 
SNRdb=[-20 -15 -10 -5 0 5 10 15 20 25 ] ; 

for nEN=l:10 

sqnrfm=0; sqnrfu=0; 
for k=l:100 

snr = 10~(SNRdb(nEN)/10) ; 

wn= randn(i,length(a) )/sqrt(snr) ; '/, noise 

al=a+wn; '/.signal plus noise 

[a_quanu,codeu,sqnr_u]=u_pcm(al,256) ; '/.call u-PCM 
[a_quanm,codem,sqnr_m]=mue_pcm(al,256,255) ; '/.call mue- 

sqnrf m=sqnrf m+sqnr_m ; 
sqnrf u=sqnrf u+sqnr_u ; 

end 

SNROm(nEN)=sqnrf m/k ; '/.bin-SNR-MUE-PCM 
SNROu(nEN)=sqnrfu/k; '/.bin-SNR-U-PCM 
end 

'/.plots 

figure; hold on; 
plot(SNRdb,SNROu, '-+') 
plot(SNRdb,SNROm, '-o') 

xlabel(' input signal-to-noise-ration in db') 

ylabel( 'output signal-to-noise-ration in db') 
legend( 'uniform PCM, 256 levels ' , 'mue-law PCM, mue=255') 
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function [a_q,snr]=u_pcm(a,n) 

'/, function to generate uniform PCM for sinwave 

'/.used in problem 3.39, CS: Haykin 

'/.Mathini Sellathurai 

n=length(a) ; 

amax=max(abs(a) ) ; 

a_q=a; 

b_q=a_q; 

d=2/n; 

q=d.*[0:n-l] ; 
q=q-((n-l)/2)*d; 
for i=l:n 

a_q(find((q(i)-d/2<= a_q) & (a_q <=q(i)+d/2) ) ) = . . . 
q(i).*ones(l,length(find((q(i)-d/2 <=a_q) & (a_q<=q(i)+d/2) ) ) ) ; 
b_q(find(a_q==q(i)))=(i-l) .*ones(l,length(find(a_q==q(i)))) ; 
end 

a_q =a_q*amax; 

snr=20*logl0(norm(a)/norm(a-a_q) ) ; 
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f unct ion [a_q , snr] =mue_pcm( s , n , mue) 

*/. function to generate mue-law PCM for sinwave 

'/.used in problem 3.39, CS: Haykin 

'/.Mathini Sellathurai 

a=max(abs(s) ) ; 

'/, mue-law 

y=(log( l+mue*abs (s/a) ) . /log( 1+mue) ) . *sign(s) ; 
[y_q,code,sqn] =u_pcm(y,n) ; 

'/.inverse mue-law 

a_q=( (( 1+mue) . ~(abs(y_q))-l) ./mue) . *sign(y_q) ; 
a_q=a_q*a; 

'/.SNR 

snr=20*logl0(norm(s)/norm(s-a_quan)) ; 



193 



Answer to Problem 3.39 
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Problem 3.40 



Matlab codes 

*/. Problem 3.40, CS: Haykin 
'/.Normalized LMS- prediction 
'/•of AR process/ speech signal 
'/. Mathini Sellathurai 

clear all 

mue=0.05; '/. step size parameter, a value between 0 ans 2 

p=2; '/. filter order 

N=10; '/. size of data 

M=l;'/, number of realizations 

'/, initializing counters 
errl=zeros(l,N-p) ; 
xhatl=zeros(l,N-p) ; 
x=zeros(l ,N) ; 

for m=l:M '/. 100 realizations 

x(i:2)= [0.1 0.2] ; 

'/.AR process 
for k=3:N 

x(k)=(0.8*x(k-l)-0.1*x(k-2))+0.1*rand(l); 
end 

'/, LMS prediction 

[err, xhat]=LMS(x,mue,p) ; 

errl=errl+err . "2; 

xhat l=xhat 1+xhat ; 

end 

plot (err 1/m, '-' ) ; 



195 



function [err, xhat]=LMS(xx,mue,p) 
'/. function Normalized LMS 
'/.p-order of the filter 
'/.mue-step size parameter 
'/.used in problem 3.40, CS : Haykin 
'/.Mathini Sellathurai 

'/, length of the data 
N=length(xx) ; 

'/, initializing weights and erros 
w=zeros(p,N-p) ; 
err=ones(l,N-p) ; 
xhat=zeros(l,N-p) ; 

'/.prediction 
1=1; 

for k=l:N-p 

h=xx(k:p+k-l) ; 
err(l)=(xx(k+p)-h*w(: ,1)) ; 
xhat(l)=h*w(: ,1); 
xxx=xx ( 1+p- 1 ) +xx ( l+p-2 ) ; 
w(: ,l+l)=w(: ,l) + (mue/xxx)*h'*err(l) ; 
1=1+1; 

end 
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Answer to Problem 3.40 



AR process (a., = -0.80, a 2 =0.10)+random noise 




0.05 1 1 1 ' — 1 1 1 1 i i 

0 100 200 300 400 500 600 700 800 900 1000 

Sample number 



Figure | : Noisy- AR- process, a 0 = -0.80, a\ = 0.10 
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Figure 2 ; Learning curves for /i = 0.0075, 0.05, 0.5 
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CHAPTER L l 
Baseband Pulse Transmission 



Problem 4.1 

(a) The impulse response of the matched filter is 

h(t) = s(T-t) 
The s(t) and h(t) are shown below: 



s(t) 
A/ 2 














T 


0 

-A/2 


T 
2 











h(t) 
A/2 












0 




T 


T 


-A/2 




2 





t 



(b) The corresponding output of the matched filter is obtained by convolving h(t) with 
s(t). The result is shown below: 



s (t) 
o 
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Problem 4.2 



(a) The matched filter of impulse response hi(t) for pulse Si(t) is given in the solution to Problem 
4.1. The matched filter of impulse response h 2 (f) for s 2 (t) is given by 

h 2 = s 2( T - f ) 



which has the following waveform: 

h 2 (t) 



A/2 



-A/2 



r/4 



3774 



t 



Fig. 1 



(b) (i) The response of the matched filter, matched to s 2 (t) and due to s^t) as input, is obtained by 
convolving h 2 (t) with s^t), as shown by 



T 

?2i(0 = f s x {x)h 2 (t-x)dz 



The waveform of the output y 2i (0 so computed is plotted in Figure 2. This figure also 
includes the corresponding waveforms of input si(t) and impulse response h 2 (t). 

(ii) Next, the response of the matched filter, matched to s^t) and due to s 2 (t) as input, is 
obtained by convolving hi(t) with s 2 (t), as shown by 



T 

12 (0 = f s 2 {x)h x {t-x)dx 

* 0 



Figure 3 shows the waveforms of input s 2 (t), impulse response h x (i), and response v 12 (0- 
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-A/2 
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2 
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2 
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o 
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T/2 



.T/2 



T 



2T 



Fig. 3 



Note that y 12(f) is exactly the negative of y 2 i(t)- However, in both cases we find that at t = 
T, both outputs are equal to zero, as shown by 



y 2 i(T) = y n (T) = o 

For n pulses s±(f) s 2 (t),...,s n (t) that are orthogonal to each other over the interval [0,T], the 
n-dimensional matched filter has the following structure: 



input 



» 


Filter matched 






to SjO) 


► 



-► Output 1 





Filter matched 






to s 2 (t) 


► 



Output 2 





Filter matched 






to s n (t) 


► 



Output n 



Fig. 4 
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Problem 4.3 



Ideal low-pass filter with variable bandwidth. The transfer function of the matched filter for a 
rectangular pulse of duration x and amplitude A is given by 

H opt (f) = sinc(fT)exp(-j7rfT) (D 

The amplitude response I H opt (f) I of the matched filter is plotted in Fig. 1(a). We wish to 
approximate this amplitude response with an ideal low-pass filter of bandwidth B. The amplitude 
response of this approximating filter is shown in Fig. 1(b). The requirement is to determine the 
particular value of bandwidth B that will provide the best approximation to the matched filter. 

We recall that the maximum value of the output signal, produced by an ideal low-pass filter in 
response to the rectangular pulse occurs at t = T/2 for BT < 1. This maximum value, expressed 
in terms of the sine integral, is equal to (2A/jt)Si(7cBT). The average noise power at the output of 
the ideal low-pass filter is equal to BN 0 . The maximum output signal-to-noise ratio of the ideal 
low-pass filter is therefore 



(SNR)' * (2A/7c) 2 Si 2 (7tBT) (2) 



0 BN, 



0 



Thus, using Eqs. (1) and (2), and assuming that AT = 1, we get 



(SNR) n o o 

0 - z .Si 2 (7cBT) 



(SNR) 0 k 2 BT 



This ratio is plotted in Fig. 2 as a function of the time-bandwidth product BT. The peak value on 
this curve occurs for BT = 0.685, for which we find that the maximum signal-to-noise ratio of the 
ideal low-pass filter is 0.84 dB below that of the true matched filter. Therefore, the " best" value 
for the bandwidth of the ideal low-pass filter characteristic of Fig. 1(b) is B = 0.685/T. 
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Problem 4.4 



The output of the low-pass RC filter, produced by a rectangular pulse of amplitude A and duration 
T, is as shown below: 

5o(0 



A(l-exp(-27i/ 0 7)) 




0 T 
The peak value of the output pulse power is 



p out = A 2 [i- exp (-27i/ 0 r)] 2 



where / 0 is the 3-dB cutoff frequency of the RC filter. 
The average output noise power is 



N of 



df 



OUt O I 9 

2 J ~l +(///„) 



The corresponding value of the output signal-to-noise ratio is therefore 



2A 

( SNR )out = T r - F [l-exp(-27i/ 0 7')] 

Differentiating (SNR) 0 with respect to f 0 T and setting the result equal to zero, we find that 
(SNR) out attains its maximum value at 



fo = 



0.2 



The corresponding maximum value of (SNR) 0Ut is 
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2 A T i 
( SNR )o >m ax = a2^AT [1 " exp( - 0 - 47l)] 

= i.62A 2 T 

For a perfect matched filter, the output signal-to-noise ratio is 
2E 

(SNR) 0jmatched = — 
iy o 

= 2 ^ T 
No 

Hence, we find that the transmitted energy must be increased by the ratio 2/1.62, that is, by 0.92 
dB so that the low-pass RC filter with / 0 = 0.2/T realizes the same performance as a perfectly 
matched filter. 

Problem 4.5 
0) Po>Pi 

The transmitted symbol is more likely to be 0. Hence, the average probability of symbol error is 
smaller when a 0 is transmitted than when a 1 is transmitted. In such a situation, the threshold X in 
Figs. 4.5(a) and (b) in the textbook is moved to the right. 

00 P[ >Po 

The transmitted symbol is more likely to be 1. Hence, the average probability of symbol error is 
smaller when a 1 is transmitted than when a 0 is transmitted. In this second situation, the threshold 
X in Figs. 4.5(a) and (b) in the textbook is moved to the left. 
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Problem 4.6 

The average probability of error is 



P e = Pi f Y (y iDd* + po j[~ fy<y l0)dx (i) 

An optimum choice of X corresponds to minimum P . Differentiating Eq. (1) with respect to k, we get: 



= Pl f Y a Id - pofya lo 

ok 



ap e 

Setting — — = 0 , we get the following condition for the optimum value of k: 
ok 



f Y& op t Id = PO 

f Y (>«)opt 10) " Pi 



which is the desired result. 
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Problem 4.7 

In a binary PCM system, with NRZ signaling, the average probability of error is 



P p = _ erfc 
e 2 



f \ 
b 




The signal energy per bit is 



E b = A 2 T b 

where A is the pulse amplitude and T b is the bit (pulse) duration. If the signaling rate is doubled, 
the bit duration T b is reduced by half. Correspondingly, E b is reduced by half. 



Let u = nEj/Nq. We may then set 



P e = 10 



-6 _ 



_ erfc(u) 
2 



Solving for u, we get 



u = 3.3 



When the signaling rate is doubled, the new value of P is 



P = _ erfc 
e 2 



e 

f \ 
U 



V* J 



= _ erfc (2.33) 
2 



= 10" 3 . 
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Problem 4.8 



(a) The average probability of error is 



P e = _ erfc 
e 2 



fib 



where E b = A 2 T b . We may rewrite this formula as 



P e = _ erfc 
2 



( K \ 



(1) 



where A is the pulse amplitude at o = >/N 0 T b . We may view o 2 as playing the role of noise variance 
at the decision device input. Let 



u = 



Eb 
N 0 



A 

G 



We are given that 



a 2 =J0" 2 volts 2 a = 0.1 volt 



P e = 10" 8 



Since P e is quite small, we may approximate it as follows: 

erfc(u) - ex P ( ~ u2) 
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We may thus rewrite Eq. (1) as (with P = 10" 8 ) 



exp(-u 2 ^ u = 10 -8 
2 



Solving this equation for u, we get 



u = 3.97 



The corresponding value of the pulse amplitude is 



A = ou = 0.1 x 3.97 
= 0.397volts 



(b) Let <T i i denote the combined variance due to noise and interference; that is 



2 2 2 

o T = cr + c { 



where o 2 is due to noise and is due to the interference. The new value of the average probability 
of error is 10" 6 . Hence 



10 



" 6 = i 
2 

_ f 



erfc 



f \ 
A 



v J 



(2) 



erfc(uT) 



where 



Ut = 



Of 



210 



Equation (2) may be approximated as (with P e = 10" ) 



eX P ( ~ U T> m 1Q -6 



Solving for Up, we get 



u T = 3.37 



The corresponding value of o 2 T is 



/A y = rO-397 ^ 
" W " [ 3.37 j 



= 0.0138 volts 2 



The variance of the interference is therefore 



o- = 



2 2 



= 0.0138 - 0.01 
= 0.0038 volts 2 
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Problem 4.9 



Consider the performance of a binary PCM system in the presence of channel noise; the receiver 
is depicted in Fig. 1. We do so by evaluating the average probability of error for such a system 
under the following assumptions: 

1. The PCM system uses an on-off format, in which symbol 1 is represented by A volts and 
symbol 0 by zero volt. 

2. The symbols 1 and 0 occur with equal probability. 

3. The channel noise w(t) is white and Gaussian with zero mean and power spectral density 
No/2. 

To determine the average probability of error, we consider the two possible kinds of error 
separately. We begin by considering the first kind of error that occurs when symbol 0 is sent and 
the receiver chooses symbol 1. In this case, the probability of error is just the probability that the 
correlator output in Fig. 1 will exceed the threshold X owing to the presence of noise, so the 
transmitted symbol 0 is mistaken for symbol 1. Since the a priori probabilities of symbols 1 and 
0 are equal, we havep = p Correspondingly, the expression for the threshold X simplifies as follows: 



X = ^ (1) 
2 

where T b is the bit duration, and A 2 ^ is the signal energy consumed in the transmission of 
symbol 1. Let y denote the correlator output: 



y = j[ Tb s(t)x(t)dt (2) 

Under hypothesis H 0 , corresponding to the transmission of symbol 0, the received signal x(t) equals 
the channel noise w(t). Under this hypothesis we may therefore describe the correlator output as 



H 0 :y = A j[ Tb w(t)dt 



(3) 



Since the white noise w(t) has zero mean, the correlator output under hypothesis H 0 also has zero 
mean. In such a situation, we speak of a conditional mean, which (for the situation at hand) we 



describe by writing 



Po = E[Y |H 0 ] = E[ j[ Tb W(t)dt 



= 0 (4) 



where the random variable Y represents the correlator output with y as its sample value and W(t) 
is a white-noise process with w(t) as its sample function. The subscript 0 in the conditional mean 
u 0 refers to the condition that hypothesis H 0 is true. Correspondingly, let c^q denote the 
conditional variance of the correlator output, given that hypothesis H Q is true. We may therefore 
write 



Oq = E[Y 2 |H 0 ] 



(5) 



W(t 1 )W(t 2 )dt 1 dta 



The double integration in Eq. (5) accounts for the squaring of the correlator output. Interchanging 
the order of integration and expectation in Eq. (5), we may write 



°o = Jo Tb EtWCtiWtafldt! dta 

(6] 

= j[ Tb Jo Tb R w (T i " *2> dt i dt 2 

The parameter R^tj - t 2 ) is the ensemble-averaged autocorrelation function of the white-noise 
process W(t). From random process theory, it is recognized that the autocorrelation function and 
power spectral density of a random process form a Fourier transform pair. Since the white-noise 
process W(t) is assumed to have a constant power spectral density of Nq/2, it follows that the 
autocorrelation function of such a process consists of a delta function weighted by Nq/2. 
Specifically, we may write 



R w (ti - t 2 ) = ^1 8(x - t x + ta) (7) 

Substituting Eq. (7) in (6), and using the property that the total area under the Dirac delta 
function 5(x - t x + t^) is unity, we get 



2 . W (8) 
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The statistical characterization of the correlator output is computed by noting that it is Gaussian 

distributed, since the white noise at the correlator input is itself Gaussian (by assumption). In 
summary, we may state that under hypothesis H Q the correlator output is a Gaussian random 
variable with zero mean and variance N 0 T b A 2 /2, as shown by 



f 0 (y) = 



y^NoTb A 



exp 



NoT^ 



(9) 



where the subscript in f 0 (y) signifies the condition that symbol 0 was sent. 

Figure 2(a) shows the bell-shaped curve for the probability density function of the correlator 
output, given that symbol 0 was transmitted. The probability of the receiver deciding in favor of 
symbol 1 is given by the area shown shaded in Fig. 2(a). The part of the y-axis covered by this area 
corresponds to the condition that the correlator output y is in excess of the threshold X defined by 
Eq. (1). Let P e0 denote the conditional probability of error, given that symbol 0 was sent. 
Hence, we may write 



J; 



exp 



N 0 TbA' 



dy 



(10) 



Define 



z = 



M>T b A 



(11) 



We may then rewrite Eq. (10) in terms of the new variable z as 



fio = — fl exp(-z 2 ) dz 



(12) 



which may be reformulated in terms of 
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complementary error function 



erfc(u) = — f 00 exp(-z 2 ) dz (13) 
/ Ju 

Accordingly, we may redefine the conditional probability of error -r e Q o~s 



f 10 " \ erfc 



A 2 T> 



(14) 



4N 0 

Consider next the second kind of error that occurs when symbol 1 is sent and the receiver chooses 
symbol 0. Under this condition, corresponding to hypothesis Hj, the correlator input consists 
of a rectangular pulse of amplitude A and duration T b plus the channel noise w(t). We may 
thus apply Eq. (2) to write 



Hi : y = A j[ Tb [A + w(t)] dt 



(15) 



The fixed quantity A in the integrand of 9 q. (15) serves to shift the correlator output from a 
mean value of zero volt under hypothesis H Q to a mean value of A 2 T b under hypothesis Hj_. 
However, the conditional variance of the correlator output under hypothesis H 1 has the same value 
as that under hypothesis H Q . Moreover, the correlator output is Gaussian distributed as before. 
In summary, the correlator output under hypothesis H x is a Gaussian random variable with mean 
A 2 T b and variance N 0 T b 2 /2, as depicted in Fig. 2(b), which corresponds to those values of the 
correlator output less than the threshold A, set at From the symmetric nature of the 

Gaussian density function, it is clear that 



Note that this statement is only true when the a priori probabilities of binary symbols 0 and 
1 are equal; this assumption was made in calculating the threshold X. 

To determine the average probability of error of the PCM receiver, we note that the two possible 
kinds of error just considered are mutually exclusive events. Thus, with the a priori probability 
of transmitting a 0 equal to p , and the a priori probability of transmitting a 1 equal to p »we find 

3J5 



that the average probability of error, P , is given by 



P. = 



(17) 



Since p o| = p ^ and p + = 1, Eq. (17) simplifies as 



or 



P. = 



01 



Po = 





f 






— erfc 


1 


A 2 T b 


(18) 


2 




No J 






Decision 
device 



T 



Choose H 1 if 
A. is exceeded 



Otherwise, 



choose H, 



o 



Figure 1 











c 






(a) 
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(b) 
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Problem 4.10 



For unipolar RZ signaling, we have 

Binary symbol 1: s(t) = +A for 0 < t < 772 
and s(t) = 0 for 772 < t < T 

Binary symbol 0: s(t) = 0 for 0 < t < T 

The a priori probabilities of symbols 1 and 0 are assumed to be equal, in which case we have 
Po=Pi = 1/2. 

To determine the average probability of error, we consider the two possible kinds of error 
separately. We begin by considering the first kind of error that occurs when symbol 0 is sent and 
the receiver chooses symbol 1. In this case, the probability of error is just the probability that the 
matched filter output will exceed the threshold X owing to the presence of noise, so the 
transmitted symbol 0 is mistaken for symbol 1. 



Energy of symbol 1 = 




Energy of symbol 0 = 0 

The conditional probability density function of the two signals is given below: 



fy(y\s=0) /K^M) 




With symbols 1 and 0 assumed to be equiprobable, the optimum threshold is 

Given that symbol 0 was transmitted, the probability of error is simply the probability that y > X, 
as shown by 



217 



P(error|0) = f Y (y\0)dy 

" — oo 




Define a new variable z as 




We then have 



P(error|0) = 4= T ,— exp(-z 2 )<fe 

2 



= -erfc 
2 



1 V^b_ 
2KN n 



= ^erfc 



2i\ 2N 0 



Similarly, P(error|l) = f f Y (y\(l)dy 



1 

-== exp 



Define z = — - , and so write 



TV, 



P(error|l) = -^fV exp(-z 2 )<fe 
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P(error|l) = 2 er ^ c 



o J 



= ^rfc 



2 J 2N, 



V 



o 



The average probability of error is therefore 
P e = P(l)P(error|l) + F(error|0)P(0) 



2t\ 2N, 



o 



(1) 



The average probability of error for on-off (i.e., unipolar NRZ) type of encoded signals is 



^erfc 
2 



V 2 V «o . 



Comparing this result with that of Eq. (1) for the unipolar RZ type of encoded signals, we 
immediately see that, for a prescribed noise spectral density Nq, the symbol energy in unipolar RZ 
signaling has to be doubled in order to achieve the same average probability of error as in unipolar 
NRZ signaling. 



Problem 4. 11 

Probability of error for bipolar NRZ signal 

Binary symbol 1 : s{i) = ±A 
Binary symbol 0: s(t) = 0 

Energy of symbol 1 = E b = A 2 T b 
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The absolute value of the threshold is A, = \jE~t, = \t]A 2 T t 



Refering to Fig. 1 on the next page, we may write 



P(error|s=-A) = * P 



exp 



-(y + *J E b ) 



dy 



Let z 



Then, 



P(error|s = -A) = 



At J K+ IE. 



erfc 



1 



erfc 



V 4 K 



Similarly, P(error|,s = +A) = P(error|5 = -A) 



P(error|.s = 0) = f°exp| -7- 



2A 



dy 



= erfc 



' l V 



2 a TV, 



The average probability of error is therefore 



P e = P(s= ± A)P(error|s= ± A) + P(s=0)P(error|s = 0) 



The conditional probability density functions of symbols 1 and 0 are given in Fig. 1: 
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Figure 1 
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D 1 1 



= Terfc 
4 



erfc 



i rb 

2jN 



erfc 



4^, 



1 K 
2^ 0 



^erfc 
4 



3 /-^ 

4A 7V n 



+ -erfc 
2 



i r* 

2a/7V 0 



v 



Problem 4.12 

The rectangular pulse given in Fig. P4.12 is defined by 
g(t) = xtc(tlT) 

The Fourier transform of g(t) is given by 



T /I 

G(f) = f cxp(-j2nft)dt 

J -T/2 



= Tsmc(JT) 
We thus have the Fourier-transform pair 
rec(t/T) ^ Tsmc(fT) 

The magnitude spectrum \G(f)\/T is plotted as the solid line in Fig. 1, shown on the next page. 

Consider next a Nyquist pulse (raised cosine pulse with a rolloff factor of zero). The magnitude 
spectrum of this second pulse is a rectangular function of frequency, as shown by the dashed curve 
in Fig. 1. 

Comparing the two spectral characteristics of Fig. 1, we may say that the rectangular pulse of Fig. 
P4.12 provides a crude approximation to the Nyquist pulse. 
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Problem 4.13 



Since P(f) is an even real function, its inverse Fourier transform equals 



p(t) = 2 j£° P(f) cos(2jift) df 



The P(f) is itself defined by Eq. (7.60) which is reproduced here in the form 



P(f) = 1 



2W 

r 



4W 



+ cos 



7t( If I - f x ) 



2W - 2f x 



0, 



0 < If I < f ! 

< f < 2W-f! 
If I > 2W-f x 



Hence, using Eq. (2) in (1): 



p(t) = — f fl cos(2jtft) df + -L f . 
* W JO 2B Jf, 



W 

(sin(2nft) 



27iWt 



2B -i 
sin(27tft) l zw - f i 



1 + cos 



2Wa 



cos(2jtft) df 



+ Aw 



( 



sin 



27lft + 



4nWt ^ 



-JT(f-fl) 

2Wa 



2jct + 7i/2Wa 



2W-fi 



4W 



sin 27ift 



2nt - 7c/2Wa 



sin(27cf 1 t) sin[2jrt(2W-f 1 )] 
4;iWt 47tWt 

1 8^(2^) + sin[27it(2W-f 1 )] 8in(2jifit) + sin^Tct^W-fit)] 



4W 



27tt - 7#Wa 



JL [sin^^t) + sin[2jit(2W-f!)]] 



2nt - 7i/2Wa 



1 _ Tit 

4wt (2Kt) 2 - (n/2Wa) 2 
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_ [sin(27iWt)cos(27iaW)] 
W 



= sinc(2Wt) cos(27iaWt) 



- (7t/2Wa) 2 



47it [(2;tt) 2 - (ii/2Wa) 2 



1-16 a 2 W 2 t 2 



Problem 4.14 



The minimum bandwidth, Rj., is equal to 1/2T, where T is the pulse duration. For 64 quantization 
levels, log 2 64 = 6 bits are required. 



Problem 4.15 



The effect of a linear phase response in the channel is simply to introduce a constant delay x into 
the pulse p(t). The delay t is defined as -l/(27t)times the slope of the phase response; see Eq. 2.171. 
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Problem 4.16 

The Bandwidth B of a raised cosine pulse spectrum is 2W - f , , where W = l/jT,. and 

I / D 

f 1 = ^(1-4) . Thus B = W(1+oQ. For a data rate of 56 kilobits per second, W = 28 kHz. 

(a) For a = 0.25, 

B = 28 kHz x 1.25 
= 35 kHz 

(b) B = 28 kHz x 1.5 

= 42 kHz 

(c) B = 49 kHz 

(d) B = 56 kHz 

Problem 4.17 

The use of eight amplitude levels ensures that 3 bits can be transmitted per pulse. 

The symbol period can be increased by a factor of 3. All four bandwidths in problem 1-12 
will be reduced to 1/3 of their binary PAM values. 

Problem 4. 18 

(a) For a unity rolloff, raised cosine pulse spectrum, the bandwidth B equals 1/T, where 
T is the pulse length. Therefore, T in this case is 1/12kHz. Quarternary PAM ensures 2 
bits per pulse, so the rate of information is 

2 bits 

| — = 24 kilobits per second. 

•(b) For 128 quantizing levels, 7 bits are required to transmit an amplitude. The 
additional bit for synchronization makes each code word 8 bits. The signal is transmitted 
at 24 kilobits/s, so it must be sampled at 

24 kbits/s „ , „ 

3 kHz. 



8 bits/sample 

The maximum possible value for the signal's highest frequency component is 1.5 kHz, in 
order to avoid aliasing. 
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Problem 4.19 

The raised cosine pulse bandwidth B = 2W - f , where W= 1/2T . For this channel, 

lb 

B = 75 kHz. For the given bit duration, VV= 50 kHz. Then, 
s 2V- B 
= 25 kHz 
Ql= 1 - f 1 /B T 
= 0.5 



Problem 4.20 

The duobinary technique has correlated digits, while the other two methods have 
independent digits. 
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Problem 4.?.1 

(a) binary sequence 
polar representation 
duobinary coder output c^ 
receiver output b^ 
output binary sequence 

(b) receiver input 
receiver output b^ 
output binary sequence 



0 0 1 
-1 -1 1 
-2 0 



-1 

0 

-1 

0 



-1 1 

0 1 

0 0 

1 -1 

1 0 



0 
-1 

0 
-1 

0 

0 
-1 

0 



0 0 

-1 -1 

0 -2 

-1 -1 

0 0 

0 -2 

-1 -1 

0 0 



We see that not only is the second digit in error, but also the error propagates. 

Problem 4.22 



(a) binary sequence b^ 
coded sequence d^ 
polar representation 
duobinary coder output c^ 
receiver output 

(b) receiver input 
receiver output 



0 0 

1 1 

1 1 

2 2 



0 
2 



0 
0 



0 1 



1 

0 

-1 

0 

1 

0 

1 



1 
1 
1 

0 

1 

0 

1 



0 

1 
1 

2 
0 
2 
0 



1 0 

0 0 

-1 -1 

0 -2 

1 0 

0 -2 

1 0 



0 
0 



-2 
0 
-2 

0 



-1 1 



0 
1 
0 
1 



In this case we see that only the second digit is in error, and there is no error 
propagation. 

Problem 4.23 



(a) The correlative coder has output 

z = y - y , 
n 'n J n-1 

Its impulse response is 
^ 1 k = 0 
h k =y k = 1 

LO otherwise. 
The frequency response is 
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H(f) = Z h k exp(-j2irfkT b ) 
k=— °o 

= 1 - exp(-j2irfT b ) 

(b) Let the input to the differential encoder be x n » the input to the correlative coder 
be y n , and the output of the correlative coder be z^. Then, for the sequence 010001101 in 
its on-off form, we have 



0 1 0 0 0 1 1 0 1 
1 1 0 0 0 0 1 0 0 1 
0 -1 0 0 0 1 -1 0 1 



Then z r has the following waveform 


























The sequence z r is a bipolar representation of the input sequence x R . 

Problem 4.24 

(a) The output symbols of the modulo-2 adder are independent because: 

1. the input sequence to the adder has independent symbols, and therefore 

2. knowing the previous value of the adder does not improve prediction of the 
present value, i.e. 

f(y n f Vl> = f(y n> • 

where y R is the value of the adder output at time nT b . The adder output 
sequence is another on-off binary wave with independent symbols. Such a wave 
has the power spectral density (from problem 4>/0) , 



S Y (f) = i- 6(f) 



2 
AT 

— sinc 2 (fT b ) . 
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The correlative coder has the transfer function 

H(f) = 1 - exp(-j2ufT b ) , 
Hence, the output wave has the power spectral density 
S z (f) = \U(f)\ 2 S Y (f) 

= [1 - exp(-j2irfT b )] [1 - exp( j2*fT b ) ] S y (f) 
= [2 - 2 cos(2TtfT b )] S y (f) 
= 4 sin 2 (TrfT b ) S y (f) 

2 

2 AT 
= 4 sin 2 (ufT b ) [|- 6(f) + -jp^ sinc 2 (fT b )] 

= A 2 T. sin 2 (irfT. ) sinc 2 (fT. ) 

D D D 

In the last line we have used the fact that 
sinUfT. ) = 0 at f = 0. 

D 




Note that the bipolar wave has no dc component. 

(Note: The power spectral density of a bipolar signal derived in part (a) assumes the use of a pulse 
of full duration T b . On the other hand, the result derived for a bipolar signal in part (d) of Problem 
3.11 assumes the use of a pulse of half symbol duration T b .) 
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Problem 4.25 



The modified duobinary receiver estimate is 




* c k + 


\-2 


» 






Va^ uxnaiy Sequence U 




1 


1 


0 


0 


1 


0 


1 


uxjjuj.dr representa&ion —1 




1 


1 


-1 


-1 


1 


-1 


1 


modified duobinary c 

K 




2 


0 


-2 


-2 


2 


0 


0 


receiver output a^ -1 


i 


1 


1 


-1 


-1 


1 


-1 


1 


output binary sequence 0 


1 


1 


1 


0 


0 


1 


0 


1 


(b) receiver input 




0 


0 


-2 


-2 


2 


0 


0 


receiver output —1 


1 


-1 


1 


-3 


-1 


-1 


-1 


-1 


output binary sequence 0 


1 


0 


1 


0 


0 


0 


0 


0 


Here we see that not only is the third digit 


in 


error, 


but 


also 


the 


error propagates. 


Problem 4 ?f, 


















uiiidry sequence u. 0 


A 

1 


1 


1 


0 


0 


1 


0 


1 


coqcu sequence U U 0 


1 


1 


0 


1 


0 


0 


0 


1 


poxar representation — \ — i — i 


1 


1 


-1 


1 


-1 


-1 


-1 


1 


modified duobinary c 0 

k 


2 


2 


-2 


0 


0 


-2 


0 


2 


receiver output b k = Ic^l 0 


2 


2 


2 


0 


0 


2 


0 


2 


output binary sequence 0 


1 


1 


1 


0 


0 


1 


0 


1 


(b) receiver input 0 


2 


0 


-2 


0 


0 


-2 


0 


2 


receiver output 0 


2 


0 


2 


0 


0 


2 


0 


2 


output binary sequence ' 0 


1 


0 


1 


0 


0 


1 


0 


1 


This time we find that only the third 
propagation. 


digit 


is 


in 


error 


, and 


there is no 
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Problem 4.?.7 

(a) Polar Signalling (M=2) 
In this case, we have 

m(t) = Z A R sinc(| - n) 
n 

where A R = ± A/2. Digits 0 and 1 are thus represented by -A/2 and +A/2, respectively. 
The Fourier transform of m(t) is 

M(f) : Z A F[sinc(| - n)] 
n 

= T rect(fT) I A R exp(-j2TmfT) 

Therefore, ra(t) is passed through the ideal low-pass filter with no distortion. 
The noise appearing at the low-pass filter output has a variance given by 




Suppose we transmit digit 1. Then, at the sampling instant, we obtain a random 
variable at the input of the decision device, defined by 




where N denotes the contribution due to noise. The decision level is 0 volts. If X > 0, 
the decision device chooses symbol 1, which is a correct decision. If X < 0, it chooses 
symbol 0, which is in error. The probability of making an error is 

0 

P(X<0) = / f x (x) dx 

_GO 

The expected value of X is A/2, and its variance is a 2 . Hence, 
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f (x) = — i— exp[ f— ] 

A /2*a 2a 

A 2 

1 0 (x - f) 

P(X<0) = _ / exp( 2^—) dx 

/2ita -°° 2a 

= \ erfc(— ^— ) 
2 2/2a 

Similarly, if we transmit symbol 0, an error is made when X > 0, and the probability 
of this error is 

P(X>0) = I erfc(— — ) 
2^20 

Since the symbols 1 and 0 are equally probable, we find that the average probability of 
error is 

P_ = \ P(X<0 I transmit 1) + \ P(X>0 | transmit 0) 
1 erfc(— — ) 



2 zr 2 



a 



(b) Polar ternary signaling 
In this case we have 

m(t) = Z A sinc(|; - n) 
n n T 

where 

A = 0, ± A. 
n ' 

The 3 digits are defined as follows 
Digit Level 

0 -A 

1 0 

2 +A 

Suppose we transmit digit 2, which, at the input of the decision device, yields the 
random variable 



X = A + N 
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The probability density function of X is 

2 

f (x) = — !— exp(- U ~ A * ) 

/2tF a 2 c 

The decision levels are set at -A/2 and A/2 volts. Hence, the probability of choosing 
digit 1 is 

a i A/2 2 

PC- f < X < §) = / — !— exp[- {X ~ A) 3 dx 
-A/2 /27 a 2a 

= 1 [erfc(— ) _ erfc(-M_)] 
2/5 a 2/5 o 

Next, the probability of choosing digit 0 is 

PCX < - |) = 1 erfcC-^— ) 
22 2/5 a 

If we transmit digit 1, the random variable at the input of the decision device is 
X = N 

The probability density function of X is therefore 

1 x 2 
f Cx) = expC- ■ 5 -,) 

/2ir a 2a 
The probability of choosing digit 2 is 

PCX >.^> = i erfcC— — ) 
22 2/2o 

The probability of choosing digit 0 is 

PCX < - ^) = \ erfcC— — ) 
22 2/2 a 

Next, suppose we transmit digit 0. Then, the random variable at the input of the 
decision device is 

X = -A + N 

The probability density function of X is therefore 
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f (x) = — - — exp[- 
A /2T o 2a 

The probability of choosing digit 1 is 

P(-4<X<4)=1 [erfc (— — ) - erfc(-^— )1 
2 2 2 2/2 a 2/2 a " 

The probability of choosing digit 2 is 

PCX > §) = \ erfc(-^-) 
2 2 2/5a 

Assuming that digits 0, 1, and 2 are equally probable, the average probability of 
error is 

P e = \ l\ erfc(— k —-) - 1 erfc(^— )] + 1 • 1 [erfc(^— )] 
32 2/5 a 2 2/5 o 32 2/5 o 

+ i - ^ [erfc(— — )] +1-1 [erfc(— ^— )] 



2/5 a 2 ' 2/5 

+ 4- • i [erfc(-A-) _ erfc( _3A__ )] +1.1 er fc(-^— ) 
32 2/5 a 2/5 a 3 2 2/5 - 



= | erfc(— ) 
3 2/5 a 



(c) Polar quaternary signaling 

In this case, we have 

a A . 3A 

A n = i 2 » * T 

and the 4 digits are represented as follows 
Digit Level 



0 
1 



3A 
2 

A 
2 



A 

+ 2 



3A 



+ 2 



Suppose we transmit digit 3, which, at the input of the decision device, yields the 
random variable: 
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The decision levels are 0, ± A. The probability of choosing digit 2 is 

A (x - ~) 

P(0 < X < A) = — — f exp[- r 2 — ] dx 

/2tt o 0 2a 

= 1 [erfcC— ) _ erfcC- 2 *— )] 
2/2 a 2/2 a 

The probability of choosing digit 1 is 

3A 2 

1 0 (x " 2> 
P(-A < X < 0) = — — / exp[- f— ] dx 

/2~jr a -A 2a 

= 1 [erfc(-2i— ) - erfcC-^—)] 
2/2 a 2/2 a 

The probability of choosing digit 0 is 

3A 2 

, -A (x - j?) 

PCX < -A) = — — I exp[- 1 ] dx 

/27 a _«> 2a 

= ^ erfcC-^— ). 
2 2/2 a 

Suppose next we transmit digit 2, obtaining 



X = | + N. 



The probability of choosing digit 3 is 

00 (x - — ) 

PCX > A) = -^ r - f exp[- j- ] dx 

/2ir a A 2a 



= \ erfcC— ^— ). 
2 2/2 a 
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The probability of choosing digit 1 is 



P(-A < X < 0) = — — f exp(- 



A 2 

0 (x - |) 



/2tT a -A 2o 



2 



•) dx 



1 [erfo(— ) - erfcC-^— )] 
2/"2o 2/2 o 



The probability of choosing digit 0 is 

A 2 

, A (x - -) 

PCX < -A) = — ■ — / expC- ~— ] dx 

/2tt a -» 2a 



= 4" erfcC-M— ). 
2 2/2 a 

Suppose next we transmit digit 1 , obtaining 



X = - | + N 



The probability of choosing digit 0 is 



PCX < -A) = 1 erfcC A ) 
2 2/2 a 



The probability of choosing digit 2 is 



PCO < X < A) = \ [erfcC— — ) - erfcC-^A— ) ] 

2/5 o 2/5 o 



The probability of choosing digit 3 is 



PCX > A) = 4 erfcC-^i— ) . 

2 2/5 a 



Finally, suppose we transmit digit 0, obtaining 



X = _ |A + n 
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The probability of choosing digit 1 is 

P(-A < X < 0) = -1 [erfc(— *— ) - e rfe(-^— )] 

2/5 a 2/2 a 

The probability of choosing digit 2 is 

P(0 < X < A) = \ [erfct-^*— ) - e rfe(-^— )] 

2/2 a 2/2 a 

The probability of choosing digit 3 is 

P(X > A) = \ erfcC-^ 4 — ) 
2 2/5 a 

Since all 4 digits are equally probable, with a probability of occurence equal to 
1A, we find that the average probability of error is 

P e = I ' 2 ' ? {erfc(— — ) - erfc(-2i— ) 
4 2 2/5 o 2/5 a 

+ erfc(-2i— ) - erfcC-^ 4 — ) 
2/2 a 2/2 a 



+ erfc(— ) 

2/2 a 

+ erfc( — -) 
2/2 a 

+ erfc(— — ) - erfc(-2 A — ) 

2/2 a 2/2 a 



3 A 
= J erfc(— 2— ) . 

2/2 a 



Problem 4.28 



The average probability of error is (from the solution to Problem 7'2S) 

P e = (1 -1) erfcC-*— ) (1) 

2/2 o 
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The received signal-to-noise ratio is 



( snr) = a2(m2 ; 1) 

R 12 o 2 



That is 



A /12CSNR), 
° " J M 2 - 1 



(2) 



Substituting Eq. (2) in (1), we get 

/3(SNR) R 

6 M / 2(M 2 -1) 

With P e = 10~ 6 , we may thus write 

10" 6 a d _ I) erfc(u) (3) 

where 

~ 3(SNR)_ 
u = ^ 

2(M -1) 

For a specified value of M, we may solve Eq. (3) for the corresponding value of u. 
We may thus construct the following table: 

M u 



2 3.37 

4 ■ 3.42 

8 3.45 

16 3.46 

We thus find that to a first degree of approximation, the minimum value of received 
signal-to-noise ratio required for P e < 10~^ is given by 

3(SNR) R 

JkSUl - C3.42) 2 

2(M -1) 

That is, (SNR) R>min «7.8 (M 2 -l) 239 



Problem 4.29 



Typically, a cable contains many twisted pairs. Therefore, the received signal can be written as 

N 

r(n) = ^v-(n) + d(n), large TV 

N 

where d(n) is the desired signal and ^v ; (n) is due to cross-talk. Typically, the v, are statistically 

i=l 

independent and identically distributed. Hence, by using the central limit theorem, as TV becomes 

N 

infinitely large, the term ^ v ( (n) is closely approximated by a Gaussian random variable for each 

;=i 

time instant n. 
Problem 4.30 

(a) The power spectral density of the signal generated by the NRZ transmitter is given by 

2 

S{f) = y|G(/)| 2 (1) 



where a is the symbol variance, T is the symbol duration, and 



G(f) = J" 1 -e- j2nft dt = Tsmc(fT) = |sinc(£| (2) 

-T/2 

is the Fourier transform of the generating function for NRZ symbols. Here, we have used the 
fact that the symbol rate R = l/T. A 2B1Q code is a multi-level block code where each block 
has 2 bits and the bit rate R = 2/T (i.e., m/T, where m is the number of bits in a block). Since 
the 2B1Q pulse has the shape of an NRZ pulse, the power spectral density of 2B1Q signals is 
given by 

a 2 2 

^2B1Q = y|^2BlQ(/)| 

where 



_ sin(27t(///g)) 
G2BIQ(/) " Jlnf 
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The factor Jl in the denominator is introduced to make the average power of the 2B1Q signal 
equal to the average power of the corresponding NRZ signal. Hence, 



'2B1Q 



(/) = 



aY sin(27r(///?)) Y 
H 7271/ J 



^-smc 2 (2(f/R)) 



(3) 



(b) The transfer functions of pulse-shaping filters for the Manchester code, modified duobinary 
code, and bipolar return-to-zero code are as follows: 

(i) Manchester code: 



1 - COS^7I-^ 



(4) 



(ii)Modified duobinary code: 



G(f) = 



1 



jj2nf 

(iii)Bipolar return-to-zero code 



COS^37t-Q - COS^Tt^ 



(5) 



G(f) = 



sin 



x sin^7t^ 



(6) 



Hence, using Eqs. (4), (5), and (6) in the formula of Eq. (1) for the power spectral density of 
PAM line codes, we get the normalized spectral plots shown in Fig. 1. In this figure, the 
spectral density is normalized with respect to the symbol variance a 2 and the frequency is 
normalized with respect to the data rate R. 

From Fig. 1, we may make the following observations: Among the four line codes displayed 
here, the 2B1Q code has much of its power concentrated inside the frequency band 
-RI2 <f<_R/2, which is much more compact than all the other three codes: Manchester code, 
modified duobinary code, and bipolar return-to-zero code. 
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0.2 0.4 0.6 0.8 



1 

f/R 



^T^^^^ I 

1.2 1.4 1.6 1.8 2 



'3 
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Problem 4.31 



The tapped-delay-line section of the adaptive filter is shown below: 




r[n] = x [n]w[n] 



d[n] = x[n] + r[n] 



Error signal e[n] = d[n] - r[n] 



w[n + l] = w[n] + [ix[n](d[n] - x [n]w[n]) 



where w[n] = [w 0 [n], ■ ■ ■, w m [n] ] 



x[n] = [x[n], x[n - 1], x[n - m] ] 



\x = learning parameter 



Problem 4.32 



(a) 



Input 



Channel 



Equalizer 



c{t) 



Ht) 



Output 



The h(t) is defined by 
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N 

h(t) = £ w k 8(t-kT) 

k=-N 



The impulse response of the cascaded system is given by the convolution sum 

N 

Pn= X w /«-; 

;=-/V 

where p n = p(nT). The Mi sample of the output of the cascaded system due to the input 
sequence {/„} is defined by 



h = P 0 h + Ya l nPk-n 



n*k 



where p^ k is a scaled version of the desired symbol I k . The summation term ^ I n Pk-n ^ 

n^k 

the intersymbol interference. 

The peak value of the interference is given by 



N 



N 



D W = X \Pn\ = X 



n=-N 
n±0 



n=-N 
n*0 



N 

E 

k=-N 



W k C n-k 



To make the ISI equal to zero, we require 



N 



Pn= X W k C n-k 



k=-N 



1, n = 0 
0, n±0 



(b) By taking the z-transform of the convolution sum 



N 



X W k C n-k 



k=-N 



and recalling that convolution in the discrete-time domain is transformed into multiplication 
in the z-domain, we get 



P(z) = H(z)C(z) 
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For the zero-forcing condition, we require that P(z) = 1. Under this condition, we have 



H(z) = 1/C(z) 



which represents the transfer function of an inverse filter. 

If the channel contains a spectral null at / = 1/2T in its frequency response, the linear zero- 
forcing equalizer attempts to compensate for this null by introducing an infinite gain at 
frequency / = 1/27". However, the channel distortion is compensated at the expense of 
enhancing additive noise: With H(z) = 1/C(z), we find that when C(z) = 0, 

H(z) = oo 

which results in noise enhancement. 

Similarly, when the channel spectral response takes a smaller value, the equalizer will 
introduce a high gain at that frequency. Again, this tends to enhance the additive noise. 

Problem 4.33 

(a) Consider Eq. (4.108) of the textbook, which is rewritten as 




Expanding the left-hand side: 




Applying the Fourier transform: 



FU_ R q (t-x)c(x)dx = F{R q (t-x)} x F{(c(x))} 



= S q (f)C(f) 
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F{q(-t)} = Q(-f) = Q*(f) 



In these three relations we have used the fact that convolution in the time domain corresponds 
to multiplication in the frequency domain. 

Putting these results together, we get 

ty/)C(/) + yC(/) = Q*(f) 
or 

(V/) + y)c(/) = Q*(f) 
which is the desired result. 

(b) The autocorrelation function of the sequence is given by 

k 

Using the fact that the autocorrelation function and power spectral density (PSD) form a 
Fourier transform pair, we may write 

PSD = F{R q (T v T 2 )} 




where F{q(t)} = Q(f) 
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Problem 4.34 

(a) The channel output is 

x(t) = s(t-t Q1 ) +a 2 s(t-t 02 ) 
Taking the Fourier transform of both sides: 

X(f) = exp(-j2nft 01 ) + o- 2 exp(-j2irft 02 )3 S(f) 
The transfer function of the channel is 

V« -IrfT 



ftl exp(-J2irft 01 ) +a 2 exp(-j2*ft 02 ) 



(b) 





Channel 

H (f) 
c 




Equalizer 

H (f) 
e 













Ideally, the equalizer should be designed so that 
H c (f) H e (f) = K Q exp(-j27rft 0 ) 

where K Q is a constant gain and t Q is the transmission delay, 
the equalizer is 

H e (f) = w 0 + w 1 ex P(-J2irfT) + w 2 exp(-ji|TrfT) 

w w 
= w 0 C1 + — exp(-j2irfT) + exp(-j4TrfT)] 



The transfer function of 



(1) 



Therefore 



H (f) 
e 



K Q exp(-j2irft 0 ) 



H (f ) 
c 



K Q exp(-j2Trft 0 ) 



<l 1 exp(-j2irft Q1 ) +Q . 2 exp(-j27rft 02 ) 
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(Kq/^) exp[-j2irf(t 0 -t 01 )] 
1 + ^ ex Pt-j2-f(t Q2 - t(J1 )] 



Since « a 1 , we may approximate H g (f) as follows 

H e (f) = ^ exp[-j27rf(t 0 - t Q1 )] {1 exp[-J2rf(t 02 - t Q1 >] 

a 2 2 

+ (— ) exp[-j4irf(t 02 - t Q1 )]} (2) 
Comparing Eqs. (1) and (2), we deduce that 

=s w 

*1 0 

'o - fc 01 " 0 

a. w„ 
1 0 

d 2 2 w 
(— ) » — 

T " fc 02 " fc 01 

Choosing K Q = a 1 , we find that the tap weights of the equalizer are as follows 

w o= 1 



w. = - 
1 CL 



1 



w„ = (— ) 
2 a 



a„ 2 
1 



248 



Problem 4.35 



The Fourier transform of the tapped-delay-line equalizer output is defined by 



Y out (f) = H(f) Xi n (f) (1) 

where H(f) is the equalizer's transfer function and Xj n (f) is the Fourier transform of the input 
signal. The input signal consists of a uniform sequence of samples, denoted by {x(nT)}. We may 
therefore write (see Eq. 6.2): 



X inW = ^ E *(f-£) (2) 

where T is the sampling period and s(t) is the signal from which the sequence of samples is 
derived. For perfect equalization, we require that 

Y out (f) = 1 for all f. 
From Eqs. (1) and (2) we therefore find that 



H(f) = ( , 

E X(f-k/T) (d) 
k 

(sequence} 

Let the impulse response/of the equalizer be denoted by {w n }. Assuming an infinite number of taps, 
we have 



H(f) = E w n exp(j27tfT) 
n = -oo 

We now immediately see that H(f) is in the form of a complex Fourier series with real coefficients 
defined by the tap weights of the equalizer. The tap-weights are themselves defined by 



Wn = Y /-MT H^PH 2 ^). n«0, +1. + 2,... 
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The transfer function H(f) is itself defined in terms of the input signal by Eq, (3). Accordingly, a. 
tapped-delay-line equalizer of infinite length can approximate any function in the frequency 
interval (-1/2T, 1/2T). 



Problem 4.36 

(a) As an example, consider the following single-parameter model of a noisy system: 
= w Q [n]x[n] + v[n] 

where x[n] is the input signal and v[n] is additive noise. To track variations in the parameter 
w Q [n], we may use the LMS algorithm, which is described by 



= w[n] + \xx[n] 



Error signal 
(d[n] - w[n]x[n]) 



= (l-[lx [n])w[n] + \xx[n] d[n] 



(1) 



To simplify matters, we assume that w[n] is independent of x[n]. Hence, taking the 
expectation of both sides of Eq. (1): 



E[w[n + l]] = (l-no x )E[w[n]] + \ir dx 



(2) 



where E is the statistical expectation operator, and 



a 2 x = E[x 2 [n]] 



r dx = E[d[n]x[n]] 

Equation (2) represents a first-order difference equation in the mean value £ , [w[n]] . For this 
difference equation to be convergent (i.e., for the system to be stable), we require that 



1 - \xo 



< 1 



or equivalently 



(i) 1 - \ia < 1, i.e., \i > 0 



2 2 
(n) - 1 +\ia x < 1, i.e., y.< — 

ov 
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Stated in yet another way, the LMS algorithm for the example considered herein is stable 
provided that the step-size parameter u, satisfies the following conditions: 



0<u.< — 



where a x is the variance of the input signal. 



(b) When a small value is assigned to u,, the adaptation is slow, which is equivalent to the LMS 
algorithm having a long "memory". The excess mean-squared error after adaptation is small, 
on the average, because of the large amount of data used by the algorithm to estimate the 
gradient vector. On the other hand, when \x is large, the adaptation is relatively fast, but at the 
expense of an increase in the excess mean-squared error after adaptation. In this case, less data 
enter the estimation, hence a degraded estimation error performance. Thus, the reciprocal of 
the parameter p, may be viewed as the memory of the LMS algorithm. 



Problem 4.37 



A decision-feedback equalizer consists of a feedforward section, a feedback section, and a 
decision device connected together as shown in Fig. 1. The feed-forward section consists of a 
tapped-delay-line filter whose taps are spaced at the reciprocal of the signaling rate. The data 
sequence to be equalized is applied to this section. The feedback section consists of another 
tapped-delay-line filter whose taps are also spaced at the reciprocal of the signaling rate. The 
input applied to the feedback section consists of the decisions made on previously detected 
symbols of the input sequence. The function of the feedback section is to subtract out that portion 
of the intersymbol interference produced by previously detected symbols from the estimates of 
future samples. 

Note that the inclusion of the decision device in the feedback loop makes the equalizer 
intrinsically nonlinear and therefore more difficult to analyze than an ordinary tapped-delay-line 
equalizer. Nevertheless, the mean-square error criterion can be used to obtain a mathematically 
tractable optimization of a decision-feedback equalizer. Indeed, the LMS algorithm can be used to 
jointly adapt both the feedforward tap-weights and the feedback tap-weights based on a common 
error signal. To be specific, let the augmented vector c„ denote the combination of the feedforward 
and feedback tap-weights, as shown by 



C n = 



W 



W 



(I)' 
n 

(2) 



(1) 
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Feedforward 




> 


section, 

A (1) 
w 









I 



Decision 
device 



Feedback 
section. 
w« 2 » 



Figure 1 



where the vectorw^ denotes the tap-weights of the feedforward section, and w^ 2) denotes the 
tap-weights of the feedback section. Let the augmented vector v„ denote the combination of input 
samples for both sections: 



V n = 



(2) 



where x n is the vector of tap-inputs in the feedforward section, and a n is the vector of tap-inputs 
(i.e., present and past decisions) in the feedback section. The common error signal is denned by 



e n = a n- C n y n 



(3) 



where the superscript T denotes matrix transposition and a n is the polar representation of the nth 
transmitted binary symbol. The LMS algorithm for the decision-feedback equalizer is described 
by the update equations: 



-(i) -(i) 

W n + 1 = W n + 



-(2) .(2) 
W *+l = W n + ^2 e n a n 



where ^ and \x, 2 are the step-size parameters for the feedforward and feedback sections, 
respectively. 
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Problem 4.3$ 



Matlab codes 

*/, Problem 4.3f, CS: haykin 
'/, Eyediagram 

'/. baseband PAM transmission, M=4 
'/, Mathini Sellathurai 
clear all 

'/, Define the M-ary number, calculation sample frequency 
M=4; Fs=20; 

'/, Define the number of points in the calculation 
Pd=500; 

'/, Generate an integer message in range [0, M-i] . 
msg_d = exp_randint(Pd, 1,M) ; 

'/, Use square constellation PAM method for modulation 
msg_a = exp_modmap(msg_d,Fs ,M) ; 

'/, nonlinear channel 
alpha=0 . 0 
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msg_a=msg_a +alpha*msg_a. ~2 

'/.raised cosine filtering 
rcv_a=raisecos_n(msg_a,Fs) ; 

'/, eye pattern 
eyescat(rcv_a,0.5,Fs) 
axis ([-0.5 2.5 -1.5 1.5]) 



function y = exp_modmap(x, Fs,M); 

'/. PAM modulation 

'/, used in Problem 4.3$ 

'/. Mathini Sellathurai 

x=x-(M-l)/2; 
x=2*x/(M-l) 

y=zeros ( length (x)*Fs, 1) ; 
p=0; 

for k=l:Fs:length(y) 
p=p+l; 

y(k: (k+Fs-l))=x(p)*ones(Fs,l); 
end 
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function out = exp_randint(p, q, r) ; 
'/. random interger generator 
'/.used for Problem 4.3g 
*/, Mathini Sellathurai 

r = [0, r-1]; 
r = sort(r) ; 
r(l) = ceiKr(i)); 
r(2) = floor(r(2)); 
if r(l) == r(2) 

out = ones(p, q) * r(l); 

return; 

end; 

d = r(2) - r(l); 

rl = rand(p, q) ; 

out = ones(p,q)*r(l) ; 

for i = l:d 

index = find(rl >= i/(d+l)); 
out(index) = (r(l) + i) * index . /index; 
end; 
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Answer to Problem 4.38 





t(s) 

Figure .3: Eye pattern for a=0.1 
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0 0.2 0.4 0.6 0.8 1 1.2 1.4 

Figure : Eye pattern for a=0.2 

Problem 4.39 

Matlab codes 

'/. problem 4.39, CS: Haykin 

'/, root raised-cosine and raised cosine sequences 
*/. M. Sellathurai 

Data=[l 0 110 0]'; 
'/. sample frequency 20 
sample_f req=20 ; 

'/.generate antipodal signal 
syms=PAM_mod(Data, sample_freq, 2); 

'/, root raised cosine pulse 

r _c_r = raisecos_sqrt(syms, sample_freq ); 

'/, normal raised cosine pulse 

r_c_n= raisecos_n(syms, sample_freq ); 

'/, plots 

t=length(r_c_r)-l; 
figure; hold on 
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plot (0: 1/20 :t/20, r_c_r) ; 
plot (0: 1/20 :t/20, r_c_n,'— '); 
xlabel( 'time' ) 

legend( 'root raised-cosine' , 'raised-cosine') 
hold off 



260 



function osyms = raisecos_n(syms , sample_freq ) 
'/, function to generate raised-cosine sequence 
*/. used in Problem 4.39, CS: Haykin 
'/.M. Sellathurai 

'/, size of data 

[l_syms, w_syms] = size(syms); 

'/. data 
R=0 . 3 ; 

W_T=[3, 3*3]; 

'/, Calculation of Raised cosine pulse 
W_T(1) = -abs(W_T(l)); 

time.T = [0 : i/sample_f req : max(W_T(2), abs(W_T(i) ) )] ; 
time_T_R = R * time_T; 



den = 1 - (2 * time_T_R) . "2; 
indexl = f ind(den"= 0) ; 
index2 = find (den ==0); 

'/, when denominator not equal to zero 
b(indexl) = sine (time_T( indexl ) ) .* cos(pi * time_T_R( indexl)) ./ den(indexl); 

'/, when denominator equal to zero, (using L'Hopital rule) 
if "isempty(index2) 
b(index2) = 0; 

end; 

b = [b(sample_freq * abs(W_T(l))+l : -1 : 1), b(2 : sample_freq * W_T(2)+1)] ; 
b=b(:)'; 

'/, filter parameters 
order= f loor(length(b)/2) ; 
bb=[]; 

for i = 1: order 

bb = [bb; b(l+i:order+i)] ; 

end; 

[u, d, v] = svd(bb); 
d = diag(d); 

index = find(d/d(l) < 0.01); 
if isempty(index) 
o = length (bb) ; 

else 
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o = index(l)-l; 

end; 
a4 = bb(l); 

ul = u(i : length(bb)-l, 1 : o); 
vl = v(l : length(bb)-l, 1 : o); 
u2 = u(2 : length(bb), 1 : o); 

dd = sqrt(d(l:o)) ; 
vdd = 1 ./ dd; 

uu = ul' * u2; 
al = uu .* (vdd * dd'); 
a2 = dd .* vi(l, :)'; 
a3 = ul(i, :) .* dd'; 

[num, den] = ss2tf(al, a2, a3, a4, 1); 

fsyms = zeros (l_syms+3*sample_f req, w_syms); 
for i = 1 : sample_freq : l_syms 
fsyms(i, :) = syms(i, :); 

end; 

'/. filtering 

for i = l:w_syms 

fsyms(:, i) = filter (num, den, fsyms(:, i)); 

end; 

osyms = fsyms(( (3 - 1) * sample_freq + 2) : (size(f syms , 1) - (sample_freq - 1)), :); 
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function osyms = raisecos_sqrt(syms , sample_f req ) 
'/, function to generate root raised-cosine sequence 
'/. used in Problem 4.39, CS: Haykin 
*/.M. Sellathurai 

'/, size of data 

[l_syms, w_syms] = size(syms); 

*/. rolloff factor 
R=0.3; 
'/. window 
W_T=[3, 3*3]; 

*/. Calculation of Raised cosine pulse 
W_T(1) = -abs(W_T(l)); 

time_T = [0 : l/sample_f req : max(W_T(2), abs(W_T(l)))] ; 

den = 1 - (4 * time_T*R) . "1; 
indexl = find(den "= 0); 
index2 = find (den ==0); 

'/, when denominator not equal to zero 
b(indexl)=( cos((l + R) * pi * time_T(indexl) )+. . . 
( s inc ( ( 1-R) *t ime_T ( indexl ) ) * ( 1-R) *pi/4/R) ) . /den ( indexl ) *4*R/ pi ; 

'/, when denominator equal to zero t=\pm T/4/alpha 
if "isempty(index2) 

b(index2)=((l+2/pi)*sin(pi/4/R)+(l-2/pi)*cos(pi/4/R))*R/sqrt(2) 
end; 

b(l) = i-R+4*R/pi; */,t=0; 

b = Cb(sample_freq * abs(W_T(l)) + l : -1 : 1), b(2 : sample freq * W T(2) + l)] • 
b=b(:)'; 

'/. filter parameters 
order= f loor(length(b)/2) ; 
bb=[]; 

for i = 1: order 

bb = [bb; b(l+i:order+i)] ; 

end; 

[u, d, v] = svd(bb) ; 
d = diag(d); 
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index = f ind(d/d(l) < 0.01); 
if is empty (index) 
o = length (bb) ; 

else 

o = index(l)-l; 

end; 
a4 = bb(l); 

ul = u(l : length(bb)-l, 1 : o); 
vl = v(l : length(bb)-l, 1 : o); 
u2 = u(2 : length(bb), 1 : o); 

dd = sqrt(d(l:o)) ; 
vdd =1 ./ dd; 

uu = ul ' * u2; 

al = uu .* (vdd * dd'); 

a2 = dd . * vl(l, :)'; 

a3 = ul(l, :) .* dd'; 

[num, den] = ss2tf(al, a2, a3, a4, 1); 

fsyms = zeros (l_syms+3*sample_f req, w_syms); 
for i = 1 : sample_freq : l_syms 
fsyms(i, :) = syms(i, :); 

end; 

'/. filtering 

for i = l:w_syms 

fsyms(:, i) = filter(num, den, fsyms(:, i)); 

end; 

osyms = fsyms (( (3 - 1) * sample_f req + 2) : (size(f syms , 1) - (sample_freq - 1)), :); 
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Answer to Problem 4.39 
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Figure 1: Raised-cosine and root raised-cosine pulse for sequence [101100] 
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CHAPTER 5 



Problem 5.1 

(a) Unipolar NRZ code. 

The pair of signals si(t) and s 2 (t) used to represent binary symbols 1 and 0, respectively are 
defined by 

s 2 (t) = 0, 0<t<T b 

where E b is the transmitted signal energy per bit and T b is the bit duration. From the 
definitions of si(f) and s 2 (t), it is clear that, in the case of unipolar NRZ signals, there is only 
one basis function of unit energy. The basis function is given by 

<M0= 0<t<T b 
H 1 b 

Then, we may expand the transmitted signals ^(0 and s 2 (t) in terms of (^(t) as follows: 
Sl (t) = jE'^ l (t), 0<t<T b 
s 2 (t) = 0, 0<t<T b 

Hence, the signal-space diagram for unipolar NRZ code is (+ jE b , 0) , as shown 



(b) Polar NRZ code. 

In this code, binary symbols 1 and 0 are defined by 
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s,(t)= + 



0<t<T, 



s 2 (t) = 0<t<T b 



The basis function is given by 



<M0 = jjr, 0<t<T b 

Then, the transmitted signals in terms of (^(f) are as follows: 
Sl (t) = jE b ^(t) 0<t<T b 

J 2(0 = ~jE b W) 0<t<T b 

Hence, the signal-space diagram for the polar NRZ code is {+ f jE~ b , -jE~ b ) as shown below: 











0 


+<E b 



(c) Unipolar return-to-zero code. 

In this third code, binary symbols 1 and 0 are defined by 

Ie~u 



0 < t < T b /2 



= 0 T b /2 <t<T b 



s 2 (t) = 0 0<t<T b 
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The energy of signal Si(t) is 



TJ2 



Ex = 



dt + 



j Odt 



T b /2 



The energy of signal s 2 (t) is zero. 
The basis function is given by 



<M0 = 



The signal-space diagram for the RZ code is as follows: 



"4>i 



o 



(d) Manchester code 

Binary symbols 1 and 0 are defined by 



MO = < 



, Q<t<T b /2 
1 1 b 



T b /2<t<T b 



s 2 (t) = { 



0<t<T b /2 



T b /2 <t<T b 



268 



The energy of signal si(t) is 



TJl 



e x = J U dt + J 



W T >J 



r/2 VV bj 



dt 



= E, 



Similarly, the energy of symbol s 2 (t) is 



E 2 ~ E b 



The basis function is given by 
sAt) 

<M0 = = 1 



The signal-space diagram of the Manchester code is thus as follows: 



• 




• 




-<E b 


0 


+<E b 


► 



Thus all the four line codes in this problem are one-dimensional. 
Problem 5.2 

The given 8-level PAM signal is defined by 



s t (t) = A f rect[ l -- T - 



The energy of signal s t (t) is given by 

T 

E l = j(Afdt 
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= A-J, A- = ±1, ±3, ±5, ±7 
The basis function is given by 



<M0 = 



Si (t) Si (t) 



jEi A t Jf 

The signal-space diagram of the 8-level PAM signal is as follows: 



— - — • • • • — 




-7Vr sJr -3Vt -Vt 


o # • • • 

Vf 3Vr sVr 7Vr 



4>1 



Problem 5.3 

Consider the signals s^t), s 2 (t), s 3 (t), and s 4 (t) shown in Fig. la. We wish to use the Gram- 
Schmidt orthogonalization procedure to find an orthonormal basis for this set of signals. 

Step 1 We note that the energy of signal s^t) is 

r T 2 
E, = sAt)dt 

C T/3 2 

= f (lfdt 



T 
3 



The first basis function (^(f) is therefore 



4>i(0 = 



JJTf, 0<t< T/3 
0, otherwise 
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S,U) 



s 2 (t> 



s 3 [t) 



S A (t) 



0 T 
3 



0 2T 
3 



<Mf> 



LL 



0 T 
3 



0 £ 
3 



(a) 



n 



0 T 2T 
3 3 



03(f) 



/3/T 



n 



0 2T r 
3 



Figure 1 



(b) 



Step 2 Evaluating the projection of s 2 (t) onto (j^O), we find that 



T 

S n = f s 2 (0<M0^ 



,r/3 



= 1 > J^< 



The energy of signal s 2 (t) is 



E 2 = fs 2 2 (t) 



r 2T/3 ? 
= f (1) dt 
J 0 



2T 
3 
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The second basis function § 2 (t) is therefore 



§ 2 (t) = 



21 



_ \JJ7f, 773<27Y3 
[ 0 otherwise 

Step 3 Evaluating the projection of s 3 (r) onto §i(t), 

T 

s 3l = J" s 3 (t)^(t)dt 



= 0 

and the coefficient s 32 equals 
c t 

s 32 = J S 3 (t)^ { (t)dt 



T 

The corresponding value of the intermediate function g t (t), with i = 3, is therefore 

g 3 (t) = 5 3 (O-5 31 0 1 (O-^32 ( I ) 2(O 

_ Jl, 2T/3<t<T 
[ 0, elsewhere 

Hence, the third basis function § 3 (t) is 

s 3 (0 



\\]{t)dt 
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JzTf, 2T/3<t<T 
0, elsewhere 



The orthogonalization process is now complete. 

The three basis functions §i(t), (^(O , and § 3 (t) form an orthonormal set, as shown in Fig. lb. In 
this example, we thus have M = 4 and TV = 3, which means that the four signals s^t), s 2 (t), s 3 (t), 
and s 4 (t) described in Fig. la do not form a linearly independent set. This is readily confirmed by 
noting that s 4 (t) = si(t) + s 3 (7). Moreover, we note that any of these four signals can be expressed 
as a linear combination of the three basis functions, which is the essence of the Gram-Schmidt 
orthogonalization procedure. 
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Problem 5.4 

(a) We first observe that s^t), s 2 (t) and s^(t) are linearly independent. 
The energy of s^t) is 

1 2 
E 1 = / (2)cit = i| 

0 

The first basis function is therefore 



s^t) 

1. 

0, 



0 <t < 1 
other wsie 



Define 



S 21 = 1 S 2 (t) V t)dt 



1 

= / (-4)(1)dt = 
0 



g 2 (t) = s 2 (t) - s 21 <t> l (t) 

f-4, 1 < t < 2 

0, otherwise 
Hence, the second basis function is 

g„(t) 



4> 2 (t) = 



/? 



g 2 (t)dt 
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) 



o, 



otherwise 



Define 



» 31 = / s 3 (t) <}> 1 (t)dt 



1 

S (3)(1)dt = 3 
0 



'32 



2 T 

' T s 3 (t) * 2 (t)dt 



= f (3)(-Ddt = -3 
1 

g 3 (t) = s 3 (t) - s 31 ^(t) - s 32 * 2 (t) 



r 



3, 
0, 



2 < t < 3 
otherwise 



Hence, the thrid basis function is 



♦ 3 (t) = 



g 3 (t) 



/? 



g 3 (t)dt 



■{ 



1, 
0, 



2 < t <3 
otherwise 



The three basis functions are as follows (graphically) 



4; 



(b) Sl (t) = 2*.,(t) 

s 2 (t) = -^(t) + 4* 2 (t) 

s 3 (t) = 3^(t) - 3* 2 (t) + 3* 3 (t) 
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Problem 5.5 



Signals s±(t) and s 2 {t) are orthogonal to each other. The energy of s^it) is 

T/2 T 

E l = J" l 2 dt+ J (-ifdt = T 

0 T/2 

The energy of s 2 (t) is 
r 

£ 2 = jl 2 dt = T 
o 

To represent the orthogonal signals s^t) and s 2 (t), we need two basis functions. The first basis 
function is given by 

W ° = ^ = 7f 

The second basis function is given by 

The signal-space diagram for and s 2 {t) is as shown below: 



i 


















0 





Problem 5.6 

The common properties of PDM and PPM are as follows: In both cases a time parameter of the 
pulse is modulated and the pulses have a constant amplitude. In PDM, the samples of the message 
signals are used to vary the duration of the individual pulses, as illustrated in Fig. la for M = 4 on 
the next page. In PPM, the position of the pulse is varied in accordance with the message, while 
keeping the duration of the pulse constant, as illustrated in Fig. lb for M = 4. 

From these two illustrative figures, it is perfectly clear that the set of PDM signals is 
nonorthogonal, whereas the PDM signals form an orthogonal set. 
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Problem 5.7 



(a) The biorthogonal signals are defined as the negatives of orthogonal signals. Consider for 
example the two orthogonal signals si(t) and s 2 (t) defined as follows: 



s 2 (t) = jE$ 2 (t) 



where (^(r) and § 2 (t) are orthonormal basis functions. The biorthogonal signals are given by 

-s^t) and -s 2 (t), which are respectively expressed in terms of the basis functions as Vis^O 

and -^E§ 2 (t). Hence, the inclusion of these two biorthogonal signals leaves the dimensionality 

of the signal-space diagram unchanged. This result holds for the general case of M orthogonal 
signals. 

(b) The signal-space diagram for the biorthogonal signals corresponding to those shown in Fig. 
P5.5 is as shown in Fig. la. Incorporating this diagram with that of the solution to Problem 
5.5, we get the 4-signal constellation shown in Fig. lb. 



<t>2 





<t>2 






r o 






> 



01 



(a) 



(b) 



Figure 1 



Problem 5.8 



(a) A pair of signals s t (t) and s k (t), belonging to an A^-dimensional signal space, can be 
represented as linear combinations of TV orthonormal basis functions. We thus write 



N 



j=i i = 1,2 



(1) 



where the coefficients of the expansion are defined by 
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i — 1,2 
7=1,2 



(2) 



The real-valued basis functions (^(r) and (j) 2 (0 are orthonormal. Hence, 



1, if i=j 
0, otherwise 



(3) 



,N 



The set of coefficients {s^} . =1 may be viewed as an iV-dimensional vector defined by 



S/ = 



U2 



UN 



i = 1,2, ...,M 



(4) 



where M is the number of signals in the setl, with M >N. The inner product of the pair of 
signal sft) and s^t) is given by 



j Si (t)s k (t)dt 



(5) 



By substituting (1) in (5), we get the following result for the inner product: 



o L . 



■ N 

■j=l 



■ N 

i=i 



dt 



N N T 
j=l 1=1 0 



(6) 



Since the §j(t) form an orthonormal set, then, in accordance with the two conditions of Eq. (3) 
and (4), the inner product of s^t) and s k (t) reduces to 



N 



\ Si {t)S k {t)dt = J^SySy 

0 y = l 
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T 

= S i S k 



(b) Consider next the squared Euclidean distance between s e - and s^, which can be expressed as 
follows: 

I s «- S *| 2 = ( S /- S *) r ( s «-Sjfc) 

T T T 

- s ; s- + s k s k - zs i s k 

T T T 

= js 2 (t)dt + js 2 k (t)dt -l^s^s^t 

0 0 0 

T 

= \{ Si {t)-s k {t)) 2 dt 

0 



Problem 5.9 

Consider the pair of complex-valued signals s^t) and s 2 (t), which are defined by 

s^t) = a n $ x {t) + a n § 2 (t) (1) 

s 2 (t) = a 2 ^ x {t) + a 22 ^ 2 {t) (2) 

The basis functions §i(t) and <J) 2 (0 are real-valued and the coefficients a n , a u , a 2 \ and a 22 are 
complex-valued. We may denote the complex-valued coefficients as follows: 

a ll = «ll +7'Pn 
«12 = ai2+7'Pl2 
a 2\ = «21 +7'P2l 
a 22 = a 22 + $22 

On this basis, we may represent the signals s^t) and s 2 (t) by the following respective pair of 
vectors: 
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r/ 




a 21 


Pll 


' §2 = 


P21 


a i2 




a 22 


Pl2 




P22 



The angle subtended between the vectors g 1 and g 2 is 

cose = „ , 

a 11 a 21 + ft n ft 21 + a i2 a 22 + P11P22 
Ja 2 n ^ 2 n + a\ 2 + (3 2 2 • 7«2iP2i + °4 + P22 



H 
s l s 2 



where Sj - 



41 



l 12 



COS0 = 



H 

s 2 s l 



and s 2 = 



< 1 



'21 



*22 



are complex vectors. Recognizing that 



we may go on to write 



j" s x {t)s* 2 {t)dt 



00 \l/2/°° ^1/2 

\\s x {t)\ 2 dt \\s 2 (tfdt 

— CO ' V — oo , 



< 1 



The complex form of the Schwarz unequality becomes 



J s l (t)s* 2 (t)dt 



< \\s x {t)\ 2 dt\\s 2 {t)\ 2 dt 
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The equality holds when si(t) and s 2 (t) are co-linear, that is, si(t) = ks 2 (t) where k is any real- 
valued constant. 



Problem 5.10 

E[XjW»(t k )] = E[(s +W )W'(t k )] 
E[s. W'(t. )] = s. . E[W'(t, )] = 0 
We also note that 

N 

W'(t > = W(t.) - Z W. <J>.(t, ) 
i-1 ilk 

We therefore have 

E[X .W' (t, )] = E[W. W' (t, )] 
J k j k 

N 

= E[W.W(t, ) - Z 4>.(t. ) E[W.W. ] 
J k i=1 i k J l 

T T 
But E[W W(t )] = E[W(t.) /■ W(t) <(>.(t)dt] = / <}..(t) E[W(t, )W(t)]dt 

3 k k 0 J o J k 

T N N 

= / * (t) • ^ 6(t-t.)dt = ♦.(t. ) 
0 J 2 k 2 j k 



B 0 , i.j 



2 

E[W.W.] 

3 ^ 




Hence, we get the final result 

E[Xj w , (tk )j = ^, j(V _i tj , v 

= 0. 
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Problem 5.11 

For the noiseless case, the received signal r(t) = s(t), 0<t<T. 

(a) The correlator output is 

T 

y(T) = jr(x)s(x)dx 
o 

T 

y(T) = \s\x)dx 
o 

= J sin (jj^jdr 
o 

0 

= T/2 

(b) The matched filter is defined by the impulse response 
h(t) = s(T-t) 

The matched filter output is therefore 

oo 

y(t) = J r(X)h(t-X)dX 



= j s(X)s(T-t + X)dX 



. J sin ^ sin ^(r-> + xy 
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Sn(T-ty 



dk- 



ijco.[iMI^] 



dk 



Since -T < k < 0, we have 



1 (8n(t-T)\f = 0 



1 T . (8n(T-t + 2k) 



-,X = 0 



X = -7 



T f8n(f-77\ T . (&n(t-Ty\ T . f&n(t + T) 
= -cos^-^ -_sm -i^ -—sm 



(c) When the matched filter output is sampled at t = T, we get 
y(T) = T/2 

which is exactly the same as the correlator output determined in part (a). 
Problem 5.12 

(a) The matched filter for signal si(f) is defined by the impulse response 
h x {t) = s^T-t) 

The matched filter for signal s 2 (t) is defined by the impulse response 
h 2 {t) = s 2 (T-t) 

The matched filter receiver is as follows 



x(t) 



Matched filter 

hit) 



Matched filter 
hit) 



x 2 



Sample at 
t = T 
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The receiver decides in favor of s 2 (t) if, for the noisy received signal, 



x(t) = s k (t) + w(t), o<^r 

k = 1, 2 

we find that x l > x 2 . On the other hand, if x 2 > x h it decides in favor of s 2 (t). If x 1 = x 2 , the 
decision is made by tossing a fair coin. 

(b) Energy of signal si(t) is given by 

T 2T 3T 

E l = j(lfdt+ J (-lfdt+ j(lfdi 
or 2T 

= 3T = E 

Energy of signal s 2 (t) is 

T/2 3T/2 5T/2 

E 2 = J (-lfdt+ I (lfdt+ $ ( 

0 T/2 3T/2 

= 3T = E 

The orthornomal basis functions for the signal-space diagram of these two orthogonal signals 
are given by 



37 

-l) 2 dt+ J (lfdt 

5T/2 



<M0 = 



Sl (t) 



3T 



and 



s 2 (t) 
J3T 



The signal-space diagram of signals s { and s 2 is as follows: 
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t *2 

V3T 



V3T 



► (]>! 



The distance between the two signal points ^(0 and s 2 (t) is 
d = V2£ = J6T 



The average probability of error is therefore 



For E/TVq, we therefore have 



P e = ^erfcQ J2^A 



= ^erfc(^) 



= 4 x 10" 



Problem 5.13 



Energy of binary symbol 1 represented by signal si(t) is 



T/2 T 
E x = J (+l) 2 dr+ J (-lfdt = r 

o r/2 

Energy of binary symbol 0 represented by signal s 2 (t) is the same as shown by 
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T/2 T 

E 2 = j (-lfdt + J ( + ifdt = T 

0 T/2 



The only basis function of the signal-space diagram is 



<M0 = 



Slit) = s^t) 

Je x ~ Jf 



The signal-space diagram of the Manchester code using the doublet pulse is as follows: 



-►4>i 



-Vf 



VF 



Hence, the distance between the two signal points is d = ijf . The average probability of error 
over an AWGN channel is given by 



P = i erfc r 

' 2 ^2jN 0 



= ^erfcf ' T 



Problem 5.14 

(a) Let Z denote the total observation space, which is divided into two parts Z and Z 
Whenever an observation falls in Z Q , we say H Q , and whenever an observation faUs i„ / 
we say Thus, expressing the risk R in terms of the conditional probability density 

functions and the decision regions, we may write 

R = c oo p o 'z/xih^'V^ 
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+ C 10 p o 'z^xih^V^ 



+ °11 Pi 'z^XJh/^I^ 



+ c oi Pi 'z/xjh/^i^ (1) 

For an N-dimensional observation space, the integrals in Eq. (1) are N-fold integrals. 

To find the Bayes test, we must choose the decision regions Z Q and in such a 
manner that the risk R will be minimized. Because we require that a decision be made, 
this means that we must assign each point x in the observation space Z to Z Q or Z^; thus 

1 = Z 0 + Z 1 
Hence, we may rewrite Eq. (1) as 



R = P 0 C 00 VxJH^'V^ + P 0 C 10 'z-Z f X(H ^ H 0 )d ^ 

o—o o—o 



+ P 1 C 11 ^Z-Z/XIH/^'"^^ + P 1 C 01 X Z f X|H <i fH 1 )d i (2) 

We observe that 

VxjH^V^ = VxiH^'"^ = 1 

Hence, Eq. (2) reduces to 
R = P 0 C 10 + P 1 C 11 

+ ^^Po^IO^OO^XIHq^^O^ + ^i^oi^h^xih ( i W 1 )],d i (3) 

The first two terms in Eq. (3) represent the fixed cost. The integral represents the cost 
controlled by those points x_ that we assign to Z Q . Since C^ Q > C Q0 and C Q1 > C^, we find 
•that the two terms inside the square brackets are positive. Therefore, all values of x 
where the first term is larger than the second should be included in Z Q because thejj 
contribute a negative amount to the integral. Similarly, all values of £ where the 
second term is larger than the first should be excluded from Z Q (i.e., assigned to Z^ 
because they would contribute a positive amount to the integral. Values of x where the 
two terms are equal have no effect on the cost and may be assigned arbitrarily. Thus the 
decision regions are defined by the following statement: If 
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P1 (C 01 - C 11 )f X|H 1 ( i» H 1 ) > P0 (C 10- C 00 )f X|H 0 ( i' H 0 ) « 

assign x to and consequently say that is true. If the reverse is true, assign x to 
Z Q and say H Q is true. 

Alternatively, we may write 

l kl" 1 (l>H i ) I' Pq^ctW 

- 0 H Q 
The quantity on the left is the likelihood ratio: 

A( X ) = — — 

" f X|H (J C i» H 0 ) 



Let 



\ p o (c io c oo ) 

= Pi (c or c ii> 

Thus, Bayes criterion yields a likelihood ratio test described by 

(b) For the minimum probability of error criterion, the likelihood ratio test is 
described by 

A <*> < ^0 

.Thus, we may view the minimum probability of error criterion as a special case of the 
Bayes criterion with the cost values defined as 

C 00 = C 11 = 0 
C 10 = C 01 



That is, the cost of a correct decision is zero, and the cost of an error of one kind is 
the same as the cost of an error of the other kind. 
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Problem 5.15 



From the signal-space diagrams derived in the solution to Problem 5.1, we immediately observe 
the following: 

1. Unipolar NRZ and unipolar RZ codes are non-minimum energy signals. 

2. Polar NRZ and Manchester codes are minimum energy signals. 

Problem 5.16 

The orfhonormal matrix that transforms the signal constellation shown in Fig. 5.11(a) of the 
textbook into the one shown in Fig. 5.11(b) is 



Q = 



1 


1 


72 


72 


1 


1 


72 


"72 



To prove this statement, we note that the constellations of Fig. 5.1 1(a) is defined by the four points 
{(a, a), (-a, a), (-a, -a), (a, -a)} . The new constellation is defined by 



/, rotate 



Qs • , which for i = 1 yields 



S l, rotate 



J_ 

72 
_L_ 

V 72 



J_ 

72 

_J_ 

72 j 



72 
0 



for a = 1. 



Similarly, s 2 



rotate 



*3, rotate 



-72 

0 



5 4, rotate 



0 

-72 



Hence, the transformation from Fig. 5.11(a) to Fig. 5.11(b) is given by Q, except for a scaling 
factor. 
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Problem 5.17 



(a) The minimum distance between any two adjacent signal points in the constellation of Fig. 
P5.17a of the textbook is 



The minimum distance between any two adjacent signal points in the constellation of Fig. 
P5.17b of the textbook is 



which is the same as d^. Hence, the average probability of symbol error using the 
constellation of Fig. P5.17a is the same as that of Fig. P5.17b. 

(b) The constellation of Fig. P5.17a has minimum energy, whereas that of Fig. P5.17b is of non- 
minimum energy. Applying the minimum energy translate to the constellation of Fig. P5.17b, 

which involves translating it bodily to the left along the 0,-axis by the amount 72a , we get 
the corresponding minimum energy configuration: 




• V2oc 



♦ 



-<2a 



0 



<2a 



Problem 5.18 



2 

Consider a set of three orthogonal signals denoted by{^ ( (r)}. =0 , each with energy E s . The 
average of these three signals is 



2 



*(0 = 55>/(0 



2 

Applying the minimum energy translate to the signal set {s i {t)}._ 
defined by 



i=0 



we get a new signal set 
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s' i (t) = s i (t)-a(t), / = 0,1,2 (1) 
The signal energy of the new set is 

e = r w))dt 

J —CO 

= [ ,y?(r)df-2f Si(t)a(t)dt+r a(t)dt 

* -co J — oo J _oo 



3^ 



The correlation coefficient p y - between the signals s\{t) and s'-{t) is given by 



2£ 



■£ (s f (0-a(0)(j/0-a(0)* 



2£ 



£ o */(0^(0^-£ o fl (0(^(0 + ^(0)* + t f a{t)dt\ (2) 



Since and sft) are orthogonal by choice, Eq. (2) reduces to 

1 , . . 

= -- for i & j 

2 J 

which is the maximum negative correlation that characterizes a simplex signal with M = 3. thus, 

2 

the signal set {s\(t)} i=() defined in Eq. (1) is indeed a simplex signal. 
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2 

To represent the signal set {s-(0};=o in geometric terms, we use the Gram-Schmidt 
orthogonalization procedure. Specifically, we first set 

*o(0 = ^ (3) 
or equivalently 
s' 0 (t) = jE<$> 0 (t) 

The projection of s\(t) unto <j) 0 0) is 
s i0 = f *i(0<MO<* 

J — oo 



1 r°° 



-4A 



The second basis function is therefore 
s i(0-s 10 <|> 0 (0 



4>i(0 = 



^7 



10 



4(Q + (Vg/2)(^) 0 (0) 
V#-(£/4) 



2 r *'i (O + ^OoC) 



J3E\ 1 2 

Accordingly, we may express s\(t) in terms of the basis functions <j) 0 (0 and (J^O) as 
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*i(0 = -^0(0 + ^0i(O 



(4) 



The remaining signal s' 2 (t) may be expressed in terms of (j) 0 (0 and §i(t) as 
JE, ,. x J3E, 



4(0 = -^yMO-^^iCO 



(5) 



Thus, using Eqs. (3) to (5), we may represent the simplex code by the following signal-space 
diagram: 





4>l 




2 


| \ 

_ I a/E/2 


\ 

\ 

\ 

\ 

\ 

\ 




r! c 


Ve 


V3E 
~2 < 


. 

1 / 

\' 





00 



Problem 5.19 

(a) An upper bound on the complementary error function is given by 



erfc(w) < 



exp(-w ) 



Jn u 

Hence, we may bound the given P e as follows: 



P e = ^erfc 



exp 



< 



2jnE b N Q 



(i) 



For large positive u, we may further simplify the upper bound on the complementary error 
function as shown here: 
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erfc( M )<^L) 

Correspondingly, we may bound P e as follows: 

p < e*P(-E b /N 0 ) 
e 2jn 

(b) For E b /N 0 = 9, we get the following results: 

(i) The exact calculation of P e yields 

P e = ^erfc(3) 
= 1.0 x 10" 5 

(ii) Using the bound in (1), we have the approximate value: 

P ~ ex P(~ 9 > 
6 6jn 



= 1.16 x 10 5 



(iii) Using the looser bound of (2), we have 



p _ exp(-9) 
2«Jn 

= 3.48 x 10" 5 

As expected, the first bound is more accurate than the second bound for calculating P e . 
Problem 5.20 

According to Eq. (5.91) of the textbook, the probability of error is over-bounded as follows: 
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M 



* = 1 

k*i 



i = 1,2, ---.M 



(1) 



where d (it is the distance between message points s t and s k . With the M transmitted messages 
assumed equally likely, the average probability of symbol error is overbounded as follows: 



M 



i=\ 



(2) 



M M 



{ a \ 



^^7iY y erfc 
2M^ ^ 



i=l k = l 



d 



ik 



The second line of Eq. (2) defines the union bound on the average probability of symbol error for 
any set of M equally likely signals in an AWGN channel. Equation (2) is particularly useful for 
the special case of a signal set that has a symmetric geometry, which is of common occurrence in 
practice. In such a case, the conditional error probability P e (m ; ) is the same for all i, and so we 
may simplify Eq. (2) as 



M 



* n 1 erfc 



k = 1 

k * i 



d 



ik 



for all i 



(3) 



The complementary error function may be upper-bounded as follows: 



erfc 



d 



ik 



2M 



, 1 

<— exp 



( d 2 ^ 
a ik 

~2Nn 



Hence, we may rewrite Eq. (3) as 



M 



l k= I v 
k*i 



( d 2 > 
a ik 

~2N n 



for all i 



(4) 
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Provided that the transmitted signal energy is high enough compared to the noise spectral density 
N 0 , the exponential term with the smallest distance d ik will dominate the summation in Eq. (4). 
Accordingly, we may approximate the bound on P e as 



M ■ 
p < min 



exp 



f d 2 ^ 
a ik 



-min 

i, k , 
i±k v 



2N r 



(5) 



where M min is the number of transmitted signals that attain the minimum Euclidean distance for 
each m ( -. Equation (5) describes a simplified form of the union bound for a symmetric signal set, 
which is easy to calculate. 
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CHAPTER 6 



Problem 6.1 

(a) ASK with coherent reception 



x(t) 



<8> 



) — - 


'o b * 




Decision 
device 













s(t) 

we raa^write 8 preSe " Ce of s^ 501 1 or symbol 0 by hypothesis or H Q , respectively, 

x(t) = s(t) + w(t) ' 
H Q : x(t) = w(t) 



where s(t) = A^os^Trf^t) , with A c = /2E^. Therefore, 
T b 

*• = / x(t) s(t) dt 
0 



If * > E b /2, the receiver decides in favor of symbol 1. If £ < E b /2, it decides in 
favor of symbol 0. 

The conditional probability density functions of the random variable L, whose value 
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is denoted by i, are defined by 



f L|1 <tll). 1 



exp[- — g ] 

N CTb 



The average probability of error is therefore, 



E /2 

P e = P o / /2 f L |0 ( * ,0)d * + Pi £ f L|1 (l,1>dX 
b — 



1 I 2 1 b 1 t £ - E K> 

ex P ( " TT~ )dl + i 1 exp[- „ . b ]d& 

E./2 /WX O b 2 - /JRX N 0 E b 



'0"b 



1 £ 

, ^ exp(- j7-pr-)dA 

/iN^ E b /2 Ob 



Ob 



= £ erfc(l /E^) 
ASK with noncoherent reception 



x(t) 



Filter 




Envelope 


matched 




detector 


to s(t) 






t=T, 



H. 



■ H 



In this case, the signal s(t) is defined by 

s(t) = A cos(2irf t + 6) 
c c 



where A p = /2 E b /T b , and 



1 

2tt ' 



f e (6) = 



0 _< 9 < 2tt 
0 , otherwise 



For the case when symbol' 0 is transmitted, that is, under hypothesis H Q , we find that 
the random variable L, at the input of the decision device, is Rayleigh-distributed : 



f Lio ulo) =or e *p<- IV> 



Ob 



0*b 



For the case when symbol 1 is transmitted, that is, under hypothesis H,, we find that the 
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random variable L is Rician-distributed: 



where I 0 (2AA c /N Q ) is the modified Bessel function of the first kind of zero order. 

Before we can obtain a solution for the error performance of the receiver, we have to 
determine a value for the threshold. Since symbols 1 and 0 occur with equal probability, 
the minimum probability of error criterion yields: 

aJ T. 21A JJl 

exp(- -|^j I C-TpS) >< 1 (1) 
* 0 U 0 Hq 

For large values of E b /N Q , we may approximate I 0 (2JtA c /N 0 ) as follows: 



2* A exp(2*A /N„) 

i r — 2.) 



c^i r c 0' 

-qv m" 

0 /4ttU /N 

c 0 



Using this approximation, we may rewrite Eq. (1) as follows: 



A (4 - A T ) n 1 /iMk 
0 H Q W 0 



Taking the logarithm of both sides of this relation, we get 

J 1 Vb 

H " T_+ 

H 0 C 

Neglecting the second term on the right hand side of this relation, and using the fact 
that 

_ c b 
E b = "T" 

we may write 
H 0 



1 /Vb 

The threshold j / — ^ — is at the point corresponding to the crossover between the 
two probability density functions, as illustrated below. 



30O cu 



I (llo) 

kT uo 




The average probability of error is therefore 



P e = p 0 P 10 + p 1 P 01 



where 



10 



= / . f. , n (A|0)dA 

/e7t u /2/2 L|U 
b b 

00 2 

= / , $r exp(- 

Vb O b 

b b 



= hexp(--ii-)] 



N 0 T b /E^T b /2/2 



6XP( - H*? 



01 



/eTt' /2/2 
b b 



f LJ1 (Ari)di 



/E~T/2/2 
b b 



= / 



2 2 2 

tor exp ( N n T h / 2 ) V-iH 

Ob Ob 0 



^b*b /2VZ SL 2 + A 2 T 2 A exp(2*A /N.) 

4* _„„c c b i c o .„ 

¥7t~ exp( - W7T72 i ' -===- d * 

Ob Ob /Utt^A /n_ 

c u 
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0 cb Ob Ob 



The integrand in Eq. (2) is the product of /2£/A c T b and the probability density function 
of a Gaussian random variable of mean A c T D /2 and variance N Q T b /4. For high values of 
E^/Nq, the standard deviation /N 0 T b /4 is much less than the threshold ✓E^/2/2. 
Consequently, the area under the portion of the curve from 0 to /E^^/2/2 is quite small, 
that is, P Q1 = 0. Then, we may approximate the average probability of error as 

P e * Po P 10 
= 2 eXp( " 

where it is assumed that symbols 0 and 1 occur with equal probability. 
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Problem 6.2 



The transmitted binary PSK signal is defined by 



s(t) = 



jE b Ht), 0<t<T b , symbol 1 
-jE b Ht), 0<t<T b , symbol 0 



where the basis function §(t) is defined by 



Ht) = — cos(2Ti/ c r) 

> i i 



The locally generated basis function in the receiver is 



iecW = /=rCos(27i/ c f + (p) 

1 b 



[cos(27t/ ( ,O cos( P - sin(27i/ J)sin(p] 



where (p is the phase error. The correlator output is given by 



T 

y = f b x(t)<p KC (t)dt 



where 



*(0 = s k (t) + w(t), k = 1,2 



Assuming that f c is an integer multiple of l/T b , and recognizing that sin(27t/ c 0 is orthogonal to 
cos(27t/ c r) over the interval 0 < t < T b , we get 

y = ± 7^ C0S( P + W 



when the plus sign corresponds to symbol 1 and the minus sign corresponds to symbol 0, and Wis 
a zero-mean Gaussian variable of variance Nq/2. Accordingly, the average probability of error of 

the binary PSK system with phase error cp is given by 
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When (p = 0 , this formula reduces to that for the standard PSK system equipped with perfect 
phase recovery. At the other extreme, when cp = ±90° , P e attains its worst value of unity. 
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£mblemjx3 

(a) The noiseless PSK signal is given by 

s(t) = A cos[2irf t+k m(t)3 
c c p 

= A cos(2*f t)cos[k m(t)] - A sin(2irf t) sin[k m(t)] 
<- u p c c p 

Since m(t) = + 1 , it follows that 

cos[k m(t)] = cos(+k ) = cos(k ) 
p — p p 

sin[k m(t)] = sin(+k ) = + sin(k ) = m(t)sin(k ) 
P — P — p p 

Therefore, 

s(t) = A co s( k ) co s( 2 irf t) - A ra(t)sin(k )sin(2irf t) (1) 
cpcc p - c 

The VCO output is 
r(t) = A v sin[2irf c t + 6(t)] 
The multiplier output is therefore 

r(t)s(t) = j A c A v cos(k p ){ sin[6 (t)] + sin[4Trf c t+6(t) ] } 

- \ A A m (t)sin(k ){cos[6(t)] + cos[4irf t+6(t)]} 

c. C V P C 

The loop filter removes the double- frequency components, producing the output 
e(t) = j A c A v cos(k p )sin[6(t)] - 1 A c A v m(t)sin(k p )cos[6(t)] 

Note that if k p = */2, (i.e., the carrier is fully deviated), there would be no carrier 
component for the PLL to track. 

(b) Since the error signal tends to drive the loop into lock (i.e., 9(t) approaches 
zero) , the loop filter output reduces to 

e(t) = - 1 A c A v sin(k p )m(t) 

which is proportional to the desired data signal m(t). Hence, the phase-locked loop may 
be used to recover m(t). 
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Problem 6.4 



(a) The signal-space diagram of the scheme described in this problem is two-dimensional, as shown 
by 



A* 



Ti 



This signal-space diagram differs from that of the conventional PSK signaling scheme in that it 
is two-dimensional, with a new signal point on the quadrature axis at A^.kVTt/2. If k is reduced 
to zero, the above diagram reduces to the same form as that shown in Fig. 8.14. 

cb) 



s(t)+w(t) 




COS(2TTf t) 

c 



The signal at the decision device input is 



I 


/ b dt 

0 


I 


Decision 
device 













H 



A f - T 

& = + — /l-k 2 T. + / w(t) cos(2irf t)dt 
-2 b Q c 
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(1) 



Therefore, following a procedure similar to that used for evaluating the average 
probability of error for a conventional PSK system, we find that for the system defined by 
Eq. (1) the average probability of error is 



P e = \ erfc(/E b (1-k 2 )/N 0 ) 

1 2 

where E. = — A T, . 
b 2 c b 

4 2 

CO For the case when P g = 10~ and k = 0.1, we get 

_4 1 
10 = - erfc(u) 

h ^ °' 9 E b 
where u = — - 

N 0 

Using the approximation 

erfc(u) = ex P ( - u2) 
/n u 

we obtain 

2 /— -4 
exp(-u ) _ 2™ x 10 u = 0 

The solution to this equation is u = 2.64. The corresponding value of E b /N Q is 

S . <2.64) 2 _ 7 74 
N Q " 0.9 " 

Expressed in decibels, this value corresponds to 8.9 dB. 
(d) For a conventional PSK system, we have 



P e = £ erfc(/E7^) 



In this case, we find that 
E 

fp = (2.64) 2 = 6.92 

0 



Expressed in decibels, this value corresponds to 8.4 dB. Thus, the conventional PSK system 
requires 0.5 dB less in then the modified scheme described herein. 
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Problem 6.5 

(a) The QPSK wave can be expressed as 

s(t) = m.,(t) cos(2irf t) + m_(t) sin(2irf t) . 
I c 2 c 

Dividing the binary wave into dibits and finding m.(.t) and m 5 (t) for each dibit: 



dibit 


11 


00 


10 


00 


10 


m^t) 


/e/t 


- /eTt 


/eTt 


- /eTt 


/EAT 


m 2 (t) 


/17t 


- /eTt 


- /e/t 


- /eTt 


- /eTt 
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Problem 6.6 

Let P eI = average probability of symbol error in to the in-phase channel 

P e Q = average probability of symbol error in to the quadrature channel 

Since the individual outputs of the in-phase and quadrature channels are statistically independent, 
the overall average probability of correct reception is 

P c = (1 - Pel) (1 " P eQ ) 

= 1 " Pel " PeQ + Pel ?eQ 

The overall average probability of error is therefore 

P e = 1 - P c 

= P el + P eQ _ P el P eQ 
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Problem 6.7 



Let r denote the received signal vector. Suppose that the signal corresponding to message point 
m,is transmitted. Then, referring to the signal-space diagram of Fig. 1, the conditional probability 
of error is given 



P e Im, = P( r li e8 in shaded region) 

= P(r lies in e=) + P(R lies in ||) 
- P(r lies in ||) 















f 








E 
N 0 


sin.iL 
M 

) 


+ 


1 
2 


erfc 




E 
N 0 


sinJL 
M 

J 



- P(r lies in ||||) 



Hence, 



p e bi! < erfc 



E . 7t 
sin_ 

N 0 M 



Assiiming that all the message points are equally likely to be transmitted, we have P p = P, 
and so 



e "e|m' 



P e < erfc 



E 
No" 



sin 



7C 

M 
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Problem 6.8 



Figures 6.10 and 6.10b of the textbook, reproduced here for convenience of presentation, depict 
the signal-space diagrams of QPSK and offset QPSK signals, respectively: 





<t>2 












►. 


i \ 






/ 1 
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/ 1 
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/ 1 
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/ 
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n ! < 




/ 0 












\ | 
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(a) QPSK 



4>i 



fa 



fa 



(b) Offset QPSK 



Figure 1 

The two parts of this figure clearly show that the signal-space structure of the offset QPSK is 
basically the same as that of the standard QPSK. They only differ from each other in the way in 
which transition takes place from one signal point to another. Accordingly, they have the same 
power spectral density, as shown by 

S(f) = E b [smc 2 (2T b (f-f c )) + smc 2 (2T b (f + f c ))] 
where T b is the bit duration and/ c is the carrier frequency. 



Problem 6.9 

(a) In vestigial-sideband (VSB) modulation, there are two basis functions: 
• The double-bandwidth sine function, defined by 



= J| sinc (f() cos ( 27l /c r ) 



(1) 



where Tis the symbol period and/ c is the carrier frequency. 
The Hilbert transform of §\(t), defined by 
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<t> 2 (0 = 




^sin(27i/ c O 



(2) 



where it is assumed that/ c > 2/T 

(Here we have made use of the Hilbert-transform pair listed as entry 1 in Table A6.4, with the 
low-pass signal m{t) set equal to J\/Tsmc 2(t/T).) 

The basis functions (1) and (2) imply the use of single-sideband modulation, which (as 
discussed in Chapter 2) is a special form of vestigial sideband modulation. We have chosen 
these definitions merely to simplify the discussion. The use of VSB substitutes a realizable 
function for the sine function that is unrealizable in practice. 

Based on the definitions of the basis functions (^(r) and (j) 2 (0 given in Eqs. (1) and (2), it may 
be tempting to choose 2/T as the symbol rate for successive transmission of binary symbols 
using binary VSB. However, such a choice of signaling destroys the orthonormality of (^(0 
and 02(0; that is, 



To maintain orthogonality of (^(r) and (^(O* successive translations of these basis functions 
must be integer multiples of l/T, as shown by 



for any integer k. 

Suppose, however, we restrict k to assume only odd integer values, and choose the carrier 
frequency f c to be an odd integer multiple of 1/27", that is, 




for = i 
for j ± i 




for ; = i 
for j ± i 



c 



I = odd integer 



(3) 



We then have the following two properties: 
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(i) ^ ^ (t)ty 2 \t - y W = 0 for a11 odd inte § er k (4) 



(ii) sin^27t/^/ - — jj = sin(27t/ c f)cos(W7t/2) - cos(27t/ c 0sin(ik/7t/2) 



= cos(2tt/ c 0 for * = odd integer 

/ = odd integer 

With such a choice, the implementation of the digital VSB transmission system is equivalent 
to a time-varying one-dimensional data transmission system, which operates at the rate of 2/7 
dimensions per second. 

(b) The optimum receiver for the digital VSB transmission system just described consists of a pair 
of matched filters, that are matched to the two basis functions ^(t) and 0 2 (O as defined in Eqs. 
(1) and (2). However, in order to conform to the design choices imposed on integer k and 
carrier frequency f c as described in Eqs. (4) and (3), the instants of time at which the two 
matched filter outputs are sampled are staggered by 7/2 with respect to each other. The two 
sequences of samples so obtained are subsequently interleaved so as to produce a single one- 
dimensional data stream as the overall receiver output. The delay by 7/2 is identical to what is 
actually done in the offset QPSK, thereby establishing the equivalence of the digital VSB 
system to the offset QPSK. 



Problem 6.10 

Assuming that modulator initially resides in a phase state of zero, we may construct the following 
sequence of events in response to the input sequence 01 101000. 



Step * 


Phase B k _i 
(radians) 


Input 
dibit 


Phase 

change 

AO, 

(radians) 


Transmitted 
phase 0, 

(radians) 


1 


0 


01 


3ti/4 


3tx/4 


2 


371/4 


10 


-71/4 


7E/2 


3 


71/2 


10 


-7E/4 


71/4 


4 


71/4 


00 


Tl/4 


71/2 
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Problem 6.11 



The output of a 7i/4-shifted QPSK modulator may be expressed in terms of its in-phase and 
quadrature components as 

s(t) = Jpcos(m/4)cos(27i/ c 0- A ^sin(m/4)sin(27i/ c 0 i = 0, 1,2, ...,7 

The different values of interger i correspond to the eight possible phase states in which the 
modulator can reside. But, unlike the 8-PSK modulator, the phase states of the 7i/4-shifted QPSK 
modulator are divided into two QPSK groups that are shifted by 7i/4 relative to each other. 

flE 

Therefore, s { (t) = /— cos(m/4) 
flE 

S Q^ = JySin(i7t/4) 
The orthonormal-basis functions for 7t/4-shifted QPSK may be denned as 
<M0 = JUs(2ti/ c 0 

<t> 2 (0 = J|sin(27i/ C r) 

Then the 7t/4-shifted QPSK signal is defined in terms of these two basis functions as 
s(t) = - s /Ecos(m/4)(j) 1 (0 - V^sin(/7t/4)(|) 2 (0 

On the basis of this representation, we may thus set up the following scheme for generating n/4- 
shifted QPSK signals: 



V£cos(-) 



-+4 



Sequence of 
input dibits 



Phase 
map 



V£sin(-) 



-+4 



MO 



7t/4-shifted 
QPSK signal 
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Problem 6.12 



A Tt/4-shifted DQPSK signal can be expressed as follows: 

s (t) = Jy-cos^-i +A(ycos(27t/ c 0-JY sin((1) *-i + A ^) sin ( 27T / c O 
flE 

= J— cos (2nf c t + $ kA + A<\> k ) 

where + A§ k = § k and § k _i is the absolute angle of symbol k-l, and A$ k is the differentially 
encoded phase change. In the demodulation process, the change in phase § k occurring over one 
symbol interval needs to be determined. 

If we demodulate the 7i/4-shifted DQPSK signal using a FM discriminator, the output of the FM 
discriminator is given by 

d[2nf c t + $ k ] 
vout(') = K Jf 



= K 



2nf c + 



dt 

= K[2nf c + ^ k ] 

where A' is a constant. In a balanced FM discriminator, the DC offset 2nf c K will not appear at the 
output. Hence, the output of the FM discriminator is KA§ t 

Problem 6.13 

The output of a Tt/4-shifted DQPSK modulator may be expressed as 

*(0 = ^cos(8 jt . 1 +Ae jfe )cos(27i/ c f)-^sin(0 t . 1 +Ae Jk )sin(27t/ c f) 



J2E 

= /— [cosQ^.jCosAG^- sin0 yi .. 1 sinA0 yS .]cos(27t/ c O 
flE 

- /— [sinO^cosAO^ + cosQ kA smAQ k ]sm(2nf c t) 
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Let I k = cos^^ + AQ k ) and Q k = sin (Q kA + AQ k ) . We may then write 
I k = cosG^cosAG^ sinG^jCosAG^. 

= 008(0^2 + A0 A .. 1 )cosA6 ;(; -sin(0 ;t . 2 + AG^cosAG^. 
from which we readily deduce the recursion 

h = 7 *-i cosA0 * -Qk-i sin A0 * 

Similarly, we may show that 

Q k = sinG^cosAG^. + cosG^.j sinAG^ 

= e^cosAe^ + ^^sinAG^ 
From the definition of I k and Q k , we immediately see that I k and Q k may also be expressed as 

h = cos0 * 
and 

Qk = sine * 

which are the desired results. 
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Problem 6.14 



-39 -35 -31 -27 -23 -19 -15 -11 



13 17 21 25 29 33 37 




Cfoss 



-39 -35 -31 -27 -23 -19 -15 -11 



13 17 21 25 29 33 37 



Q 



tiv _ o-oaxI G-CCb^v* r£ \/ 3 4 1^jdcUK\ 
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Problem 6.15 

The transmission bandwidth of 256-QAM signal is 
2R, 

B = b 



log 2 M 

where R b is the bit rate given by \IT b and M = 256. Thus 

= 2(l/T b ) _ 2 _ 1 
256 log 2 256 16 T b &T b 

The transmission bandwidth of 64-QAM is 

= 2(i/r,) = j_ = j_ 
64 log 2 64 sr fc 47, 

Hence, the bandwidth advantage of 256-QAM over 64-QAM is 

J 1 _ 1 

4T b ST b ST b 

The average energy of 256-QAM signal is 
_ 2(M-l)ff 0 _ 2(256- l)E 0 

^256 - o - o 



= 170E 0 

where is me energy of the signal with the lowest amplitude. For the 64-QAM signal, we have 
E 64 = ^f^o = 42£ 0 

Therefore, the increase in average signal energy resulting from the use of 256-QAM over 64- 
QAM, expressed in dBs, is 



10IO8 l0 l42^> 101 °8,0(4) 



= 6dB 
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Problem 6.16 

The probability of symbol error for 16-QAM is given by 



* jerfc 



3E n 



\2{M-\)N Q 



Setting P e = 10, we get 



10 3 = 2 



erfc 



P^av 



30JV r 



Solving this equation for E av /NQ, 



'av = 58 



N 0 = 17.6dB 



The probability of symbol error for 16-PSK is given by 



P e = erfc(^J^sin(7t/M) 



Setting P e = 10" 3 , we get 



10 3 = erfc(^J-sin(7i/16) 



Solving this equation for E/N 0 , we get 



= 142 = 21.5dB 



Hence, on the average, the 16-PSK demands 21.5 - 17.6 = 3.9 dB more symbol energy than the 
16-QAM for P e = 10" 3 . 



Thus the 16-QAM requires about 4 dB less in signal energy than the 16-PSK for a fixed N 0 and P e 

= 10" 3 , However, for this advantage of the 16-QAM over the 16-PSK to be realized, the channel 
must be linear. 
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Problem 6.17 

(a) An M-ary QAM signal is defined by 



s k(t) = ^a k cos(2nf c t)- ^a k sm(2nf c t) (1) 

We can redefine the M-ary QAM signal in terms of a general pulse-shaping function g(t) as 

= a k g(t-kT)cos(2nf c t)-b k g(t-kT)sm(2nf c t) (2) 
The M-ary QAM signal s(t) for an infinite succession of input symbols can be expressed as 



J (0 = E 



= E {^(?-^)cos(27t/ c 0- ^g(r-*r)sin(27i/ c f)} 

A:=-oo 



= Re E («k + jb k )8(t-kT)e c 



= Re jA rf (f-tr)« (3) 

U=-oo J 

where is a complex number defined by 
A k = a k + J b k 

By multiplying Eq. (3) by expi^nfJcT) x exp^nfJcT), we get 



*(0 = *J E A k g(t-kT)cxp(-j2nf c kT)Gxp(j2nf c t) 

U=-°° 



= Re JA^-^) (4) 

U=-~ J 
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where A k = A k exp(j2nf c kT) 
g(t) = g(t)exp(j2nf c t) 

The scalar A* is a rotated version of the complex representation of the Ath transmitted signal. 

Equation (4), representing a QAM signal, appears to be carrierless, therefore, it is equivalent 
to a CAP system. 

(b) A CAP signal is defined as 



s(t) = Re ^A k ~g(t-kT) 

U=-o« 



Re £ A ke x V (j2nf c kT)g(t-kT)exp(j2nf c (t-kT)) 

U=-oo 



Rcl A k ^g(t-kT)exp(j2nf c t) 

L k=-°o 



= X a k g(t-kT)cos(2nf c t)- £ b k g{t - kT)sm(2nf c t) (5) 

k=-°° k=-°° 

l2E 

Now the pulse shaping functions of CAP signal, g(t - kT), may be replaced by /— for 0<t<T, 
and the formulaton in Eq. (5) can be rewritten as 



*W = X ^a k cos(2nf c t)- ^b k sin(2nf c t) (6) 

k=-°° 

The kth signal of the signal s(t) defined in Eq. (6) is given by 

s k( f ) = ffa k cos(.2nf c t)-^b k sm(2nf c t), 0<t<T (7) 
The signal formulation given in Eq. (7) is recognized as the M-ary QAM signal of Eq. (1). 
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Problem 6.18 



The CAP signal can be expressed as 



s(t) = ^a nP {t-nT)- J^bJit-nT) 



where pit) is the Hilbert transform of the pulse pit), and A n = a n + jb n . The CAP signal sit) can 
be written in the equivalent form: 



sit) 



^a^it-nT) 



* Pit) 



"£b n bit-nT) 



* Pit) 



where 8(0 is the delta function, and the star denotes convolution in the time domain. Hence, the 
power spectral density of sit) is 



S s if) = ylW)| 2 + y|£(/)| : 



2 2 

where a a and a b are the variances of symbol a k and b k , respectively, where pit) and 
p(f)ssfcP(/). Noting that = P(f), we thus have 



2 2 



S s if) = -H—°\p {f) \< 



Next, noting that 



2 2 2 1 -r-! . 2 , 2. 

i=l 



we finally get 



S s if) = vAPiff 



323 



Problem 6. 19 

From the defining equations (6.74) and (6.75) of the textbook, we have 

pit) = g(/)cos(27t/ c O 

and 

p(t) = git)smi2nf c t) 

Applying the Fourier transform to Eqs. (1) and (2), we get 

Pif) = \iGif-f c ) + Gif + f c )] 

and 

Pif) = jjlGif-f c )-Gif + f c )] 

Accordingly, we may determine pit) and pit) by proceeding as follows: 

• Given G(f), use Eqs. (3) and (4) to evaluate Pif) and Pif) . 

• Using the inverse Fourier transform, compute pit) = F l [Pif)] and pit) 
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Problem 6.20 

For binary FSK, the two signal vectors are 



0 - 1 

0 



where E b is the signal energy per bit. 
with the observation vector 



The inner products of these two signal vectors 



x = 



r i 
X 1 



K 2 - 1 



are as follows, respectively, 
U.fi 2 ) = ^ x 2 

where (x, = x 1 ^ for is 1,2. The condition 



T T 
x 1 &! > x 1 s 2 



is therefore equivalent to 



fib x l > x 2 



Cancelling the common factor VeJ>, we get 



xi > x 2 



which is the desired condition for making a decision in favor of symbol 1. 
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Problem 6.7.1 



The bit duration is 



T = — 7 — = 0.4 us" 

2.5x10 Hz 



■The signal energy per bit is 

E b -1 A c T b 

= \ (10" 6 ) 2 x 0.4 x 10" 6 = 2 x 10" 19 joules 

(a) Coherent Binary FSK 

The average probability of error is 



1 



P e = 2 erfe(/E^/2M 0 ) 



= \ er fc(/2x1 0" 1 9 /4x1 0 -20 ) 



1 erfc(/5) 



Using the approximation 

erfc(u) - ex £ ( -» 2) 
/ir u 

we obtain the result 



P = lexp( z 5j. = - 3 
e 2 /5tt 



(b) MSK 



P e = erfc(/E b /N 0 ) 



= erfc(/T5") 
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exp(-IO) 

/iChr 



0.81 x 10" 5 



(c) Noncoherent Binary FSK 



p 1 , V 

P e = 2 eXp( " a£> 



^ exp(-5) 



3.37 x 10 -3 



Problem 6.22 



(a) The correlation coefficient of the signals s Q (t) and s^t) is 



T. 

/ s 0 (t) Sl (t)dt 



T o 1/2 T. . 1/2 

[/ D s^(t)dt] [/ 0 sr(t)dt] 
0 u 0 ' 



T 

A 2 / b cos[2Tr(f + ^AOt] cos[2ir(f - ^Af)t] dt 
c q c 2 c 2 



[1 A 2 T ] 1/2 [lA 2 T ] 1/2 
L 2 Vb J L 2 c b J 



T 

4- / [cos(2itAft) + cos(4irf t)] dt 
T b 0 C 



sin(2irAfT ) sin(4irf T w ) 

1 r £_ C D ■■ 



~ 2ttT. Af 2f 

b c 

Since f » Af, then we may ignore the second term in Eq. (1), obtaining 

sin(2irAfT ) 
p = 2^ Af = sinc(2AfT b ) 



(b) The dependence of P on Af is as shown in f±o, i. 
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-0.216 



Fig. 1 
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s Q (t) and s^t) are othogonal when p = 0. Therefore, the minimum value of Af for which 
they are orthogonal, is 1/2T. . 

D 

(c) The average probability of error is given by 

E b = 1 erfc(/E b (1-p)/2N 0 ) 

The most negative value of p is -0.216, occuring at Af = 0.7/T.. The minimum value of P 
is therefore 

P e,min 4 erfc( /0 - 608 VV 

(d) For a coherent binary PSK system, the average probability of error is 

P e = \ erfcC/E^) 

Therefore, the E b /N Q of this coherent binary FSK system must be increased by the factor 
1/0.608 = 1.645 (or 2.16 dB) so as to realize the same average probability of error as a 
coherent binary PSK system. 

Problem 6.23 

(a) Since the two oscillators used to represent symbols 1 and 0 are independent, we may 
view the resulting binary FSK wave as the sum of two on-off keying (00K) signals. One 00K 
signal operates with the oscillator of frequency f^. The second 00K signal operates with 
the oscillator of frequency f^. 

The power spectral density of a random binary wave X^t), in which symbol 1 is 
represented by A volts and symbol 0 by zero volts, is given by (see Problem *|«/0) 

2 

2 AT - 

s x/ f) = f 6(f) + ~*r~ sinc2 ( f y 

•where T b is the bit duration. When this binary wave is multiplied by a sinusoidal wave of 
unit amplitude and frequency f + Af/2, we get the first 00K signal with 

A = ^2E. /T. 

b b 

The power spectral density of this 00K signal equals 
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The power spectral density of the random binary wave X 2 U) = X^t), in which symbol 
1 is represented by zero volts and symbol 0 by A volts, is given by 

S„ (f) = S Y (f) 

When X 2 <t) is multiplied by the second sinusoidal wave of unit amplitude and frequency 
f c - Af/2, we g e t the second 00K signal whose power spectral density equals 

s 2 (n ^[s^f- f c + ^) .s^f + f c -|£)] 

The power spectral density of the FSK signal equals: 
S psK (f) = S^f) + S 2 (f) 

■ ^ [6(f - f c - r> * •« ♦ f = * r> ♦ ^ -'.♦!*>♦ 5(f * f c - r» 

E 

+ ^ {sinc 2 [T b (f - f c - + sinc 2 [T b (f + f e + 

+ sinc 2 [T b (f - f c + + sinc 2 [T b (f + f c - f-)D 

This result shows that the power spectrum of this binary FSK wave contains delta 

functions at f = f + Af/2. 

c — 

(b) At high values of x, the function sinc(x) falls off as 1/x. Hence, at high 

p 

frequencies, S falls off as 1/f . 
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Problem 6.24 




^J hB ^ imilar t tieS b6tWeen ° ff3et QPSK and MSK are that b °th have a half-symobl delay 
probTiUtl oTe^.^ C ° mP ° nentS ° f ^ data S ^ a " d both ha - the « 

QPSK Ir! ^n fer ^ C6S ^!^ 6n the tW ° techni ^ ues are : (D the basis functions for offset 
are aSLu, ™ ?! 71 3 rectan S le function, while the basis functions for MSK 

! „ ^ 0ld ;« U u ?J ed by half 3 COsine pulse « and (2) offset QPSK is a form of phase 
modulation while MSK is a form of frequency modulation. 
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Problem 6.26 



For coherent MSK, the probability of error is 



P e = erfc(/E b /N 0 ) , 
while for noncoherent MSK, (i.e., noncoherent binary FSK) 



e 2 



1 r 

P exp(- w ~) . 



To maintain P = 10 3 for coherent MSK, ~ 
6 N 0 
symbol error for noncoherent MSK, 



=9.8. To maintain the same probability of 



jj— = 21.6, which is an increase of 3.4 dB. 



Problem 6.27 



(a) 



0 0 I 



0 o o 



b x z — n 21 0 



7r 



vrKt> 



(b) 
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Problem 6.28 



(a) The Fourier transform of h(t) is given by (using entry 5 of the Fourier-transform pairs Table 
A6.3) 



H(f) ~ — — — exp 
a *Jn/a 



-71 x 



2 A 



V 



n/a 2 j 



= exp-(/ 2 a 2 ) 



(1) 



Substituting a = (J\og2/2/W) into (1), we get 



utf\ i A log 2 1 

H(f) = exp( -/ -J- x — 

2 W 



= exp 



log2f/ 



2 \W 



(2) 



Let/ 0 denote the 3-dB cut-off frequency of the GMSK signal. Then, by definition, 
\H(f 0 )\ = ^1^(0)1 

= _L 

= 42 

Hence, from Eq. (2) it follows that 



exp 



2 KWJ J 72 



or 



exp(log2(-"| | = 2 



Taking the logarithm of both sides, we readily find that 
fo = W 
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The 3-dB bandwidth (cut-off frequency) of the filter used to shape GMSK signals is therefore 
W. 



(b) The response of the filter to a rectangular pulse of unit amplitude and duration T centered on 
the origin is given by 

T/2 

g{t) = J" h{t-x)dx 
-T/2 




(3) 



Let k = 



n(t - t) 



, and 



a 



dk = — dx 
a 



Hence, we may reqrite Eq. (3) as 




k, 



(4) 



k 



where k, = 



n(t + T/2) 



and 



a 



n(t-T/2) 



a 



Equation (4) is finally rewritten as 




= -^rf^ + ^erf^) 
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= -l[l-erfc(^ 2 )] + i[l-erfc(A: 1 )] 



= ^erfc(£ 2 )-^erfc(£j) 




GMSK signal 
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The GMSK signal, displayed in the bottom waveform, is very similar to that of the MSK signal in 
Fig. 6.30, both of which are produced by the input sequence 1101000. This objective is indeed the 
idea behind the GMSK signal. 

Problem 6.30 

Comparing the standard MSK and Gaussian-filtered GMS signals, we note the following: 

(a) Similarities 

• For a given input sequence, the waveforms produced by the MSK and GMSK modulators 
are very similar, as illustrated by comparing the GMSK signal displayed in the solution to 
Problem 6.29 and the corresponding MSK signal displayed in Fig. 6.30 of the textbook for 
the input sequence 1101000. 

• They both have a constant envelope. 

(b) Differences 

The use of GMSK results in a slight degradation in performance compared to the standard 
MSK for a time-bandwidth product WT b = 0.3. However, the GMSK makes up for this loss 
in performance by providing a more compact power-spectral characteristic. 
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Problem 6.31 



In the binary FSK case, the transmitted signal is defined by 



Si(t) = ' 



E 

cos(27tfit), 

T b 

0, 



0 < r < T b 
elsewhere 



(1) 



where the carrier frequency fj equals one of two possible values fj and f 2 . The transmission of 
frequency fj represents symbol 1, and the transmission of frequency f 2 represents symbol 0. 
For the noncoherent detection of this frequency-modulated wave, the receiver consists of a pair 
of matched filters followed by envelope detectors, as in Fig. 1. The filter in the upper channel 
of the receiver is matched to ^2/T b cos(2jtfjt) and the filter in the lower channel is 
matched to ^2/T B cos(27if 2 t),0 < t < T b . The resulting envelope detector outputs 
are sampled at t = T b , and their values are compared. Let l x and 1 2 denote the envelope 
samples of the upper and lower channels, respectively, Then, if ^ > 1 2 , the receiver decides in 
favor of symbol 1, and if lj < 1 2 it decides in favor of symbol 0. 



Suppose symbol 1 or frequency fj is transmitted. Then a correct decision will be made by the 
receiver if lj > 1 2 . If, however, the noise is such that lj < 1 2 , the receiver decides in favor of 
symbol 0, and an erroneous decision will have been made. To calculate the probability of error, 
we must have the probability density functions of the random variables Lj^ and L 2 whose 
sample values are denoted by lj and 1 2 , respectively. 

When frequency fj_ is transmitted, and there is no synchronism between the receiver and 
transmitter, the received signal x(t) is of the form 



x(t) = 



2Ey 



2E V 



cos(27cf!t + 8) + w(t) 



(2) 



cos 6 cos(27cf 1 t) - 



2E> 



sin 9 sin(27if 1 t) + w(t), 0 < t < T b 



Let 
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x ci 



-Jt 



T b 



x(t) 



2 

T b 



cos(27ifit)dt, i = 1,2 



(3) 



x(t). 



Filter matched to 
JL cos (2* /,*) 

0<i<r /) 



Filter matched to 
£ cos(27r/ 2 f) 

0<t<T b 





Envelope 




detector 



Sample at time 

t = n 



— > 


Envelope 




detector 















Comparison 
device 


> If/, >/ 2 
choose 1 

If/, </ 2 
choose 0 


i 


i 




/ 2 





Sample at 
time t = T h 



Figure 1 



and 



A sin(27tf i t)dt, i = 1,2, (4) 
T b 

The 2^ and x si i=l,2, define the coordinates of the received signal point. Note that, although 
each transmitted signal s^t), i=l,2, is represented by a point in a two-dimensional space, the 
presence of the unknown phase 6 makes it necessary to use four orthonormal basis functions 
in order to resolve the received signal x(t). With the received signal x(t) having the form shown 
in Eq. (1), we find that the output of the upper channel in the receiver of Fig. 1 equals 




where 



x ci = \Sb cos e + w ci (6) 

and 

x si = " fib sin G + w sl (7) 
On the other hand, the corresponding value of the lower channel output is 



l 2 " Y A c2 ^ A s2 
where 

x c2 = w c2 

and 

x s2 = w s2 

The w d and w si , i=l,2, are related to the noise w(t) as follows 



i r~2 r~ (8) 

1q = yx„ 0 + x„ w 



(9) 



(10) 



w ci 



N T b 



2 cos(27cfit)dt, i = 1,2 



and 



w si 



— sin(27ifit) dt, i = 1,2, < 12 ) 
T b 



Accordingly, w^ and w si , i=l,2, are sample values of independent Gaussian random variables 
of zero mean and variance Nq/2. 

When symbol 1 or frequency f x is transmitted, we see from Eqs. (9) and (10) that x c2 and x s2 
are sample values of two Gaussian and statistically independent random variables, and 
Xg 2 , with zero mean and variance Nq/2. Accordingly, the lower channel output 1 2 , related to x c2 
and Xg 2 by Eq. (8), is the sample value of a Rayleigh-distributed random variable L 2 . We may 
thus express the conditional probability density function of L 2 , given that symbol 1 was 
transmitted, as follows: 339 



*L2 lida 



1) = ^ exp 
N 0 



_ 
No" 



1 2 ^0 



(13) 



Again tinder the condition that symbol 1 or frequency f x is transmitted, we see from Eqs. (6) 
and (7) that x cl and x gl are sample values of two Gaussian and statistically independent 
random variables, and X^, with mean values equal to ^E^ cos 9 and Je^ sin 0, 
respectively, and variance Nq/2. Therefore, the joint probability density function of and 
X sl , given that symbol 1 was transmitted and that the random phase 8 = 9, may be expressed 
as follows 



* x cl> x sl I 1 ' ®( x cl> x sl I e ) " -h- ex P 

7lN 0 



1 

N 0 



(x cl -^ b ~cos9) 2 + (x sl + ^ b ~sin9) 2 



414) 



Define the transformations 



and 



x cl = lj cos 



x sl = l x sin 



)15) 



(16) 



where ^ = tan" 1 (x gl /x cl ), with 0 < y\ < 2rc. Then, applying this transformation and following 
a procedure similar to that described in Section 5.12, we find that the upper channel output 
l x is the sample value of a Rician-distributed random variable L v Hence, the conditional 
probability density function of L ls given that symbol 1 was transmitted and that the random 
phase 0 = 9, is given by the Rician distribution 



fLibdi lie) - 



r 


fLi.^li.etti. V 


1, 9)dy 






f \ 




f \ 


21i 


exp 




Io 




N 0 


{ N o J 


N o J 



(17) 



l! > 0 



where I 0 (21 1 VE b /N 0 ) is the modified Bessel function of the first kind of zero order. Since Eq. 
(17) does not depend on 9, which is to be expected, it follows that the conditional probability 
density function of L lf given that symbol 1 was transmitted, is 
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futdi ll) = ^iexp 



ll 2 + E, 



N, 



o 



2i lV i^ 



li > 0 



(18) 



Note that by putting = 0 and recognizing that I 0 (0) = 1, Eq. (18) reduces to a Rayleigh 
distribution. 



When symbol 1 is transmitted, the receiver makes an error whenever the envelope sample 1 2 
obtained from the lower channel (due to noise alone) exceeds the envelope sample l x obtained 
from the upper channel (due to signal plus noise), for all possible values of 1^ Consequently, 
the probability of this error is obtained by integrating f L2 1 ^2 1 1) with respect to 1 2 from l x to 
infinity, and then averaging over all possible values of l v That is to say, 

p 01 = P(l 2 >li I symbol 1 was sent) 



(19) 



where the inner integral is the conditional probability of error for a fixed value of l v given that 
symbol 1 was transmitted, and the outer integral is the average of this conditional probability 
for all possible values of 1^ Since the random variable L 2 is Rayleigh-distributed when symbol 
1 is transmitted, the inner integral in Eq. (19) is equal to expHj/2/fy). Thus, using Eq. (18) in 
(19), we get 



'01 



exp 



21i + E b 
Nn 



Define a new variable v related to by 



v = 



21, 



N, 



dl. 



(20) 



(21) 



Then, changing the variable of integration from ^ to v, we may rewrite Eq. (20) in the form 



5 oi = 2 GXP 



f \ 



2N 



0 



Jf 



v exp 



v 2 + a 2 



V 



I 0 (av)dv 



(22) 



where a = VE^j/Nq. The integral in Eq. (22) represents the total area under the normalized form 
of the Rician distribution. Since this integral must be equal to one, we may simplify Eq. (22) 
as 
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p " = ^ exp (-2^rJ 



(23) 



Similarly, when symbol 0 or frequency f 2 is transmitted, we may show that p l0 , the probability 
that 1 1 > l 2 and therefore the probability that the receiver makes an error by deciding in favor of 
symbol 1, has the same value as in Eq. (23). Thus, averaging p 10 an d Pq\, we find that the average 
probability of symbol error for the noncoherent binary FSK equals 

p < = H"ft) < 24 > 

which is exactly the same as that in Eq. (6.163) in the textbook. 

Comparing the effort involved in the derivation of Eq. (24) presented in this problem with that in 
deriving Eq. (6.163), we clearly see the elegance of the approach adopted in the textbook. 
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Problem 6.32 
Let 

x(t) = A cos(2irf t + 6) 
c c 

= A cos(2irf t) cos 8 - A sin(2irf t) sin 6 
c c c c 

The output of the square-law envelope detector in Fig. P B e Z , sampled at time t=T, is 
given by 

T T 
y(T) = [/ x(t) cos(2irf t) dt] 2 + [/ x(t) sin(2irf t) dt] 2 
0 ° 0 

This may be written as 

T T 

y(T) = / / x(t.)x(t.)[cos(2Trf t 1 )cos(2 l rf t.) + sin(2irf t 1 )sin(2irf t.)]dt,dt_ (1) 
qq!*: Cl C2 Cl C 2 1 2 



Put t 1 = t, and t 2 



t+x. This transformation is illustrated below: 

T 



*2 






T 






0 


T 



0 



Then, we may rewrite Eq. (1) as follows 
T T-t 

y(T) = / / x(t)x(t+T) [cos(2irf t)cos(2Tif t + 2irf t) 
0 -t c c c 

+ sin(2nf t)sin(2irf t + 2irf t) ] dt dx 
c c c 

However, 

cos(2irf t)cos(2nf t + 2uf x) + sin(2irf t)sln(2nf t + 2uf x) 
** c c c c c 

Therefore, we may simplify Eq. (2) as follows 
T T-t 

y(T) = / / X (t) x(t+x) cos(2Trf x) dx dt 
0 -t c 

T T-t 

= 2 / / x(t) x(t+x) cos(2irf x) dx dt, 0 < x < T 
0 0 c — — 



cos(2irf x) 
c 



(2) 



(3) 
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Define 

T-t 

R Y (t) = / x(t) x(t+x) dt 0 < T 

A 0 

Then, we may rewrite Eq. (3) in terras of R x (t) as follows 
T 

y(T) = 2 / R y (t) cos(2Trf x) d T 
0 c 

= 2 S y (f ) 
X c 

where 

T 

S„(f) = / R Y (x) cos(2irf x) dx 
X 0 X c 



Equation (3) is the desired result. 



Problem 6.33 

(a)bk 1100100010 

dk.! 1110110100 

d k 11101101001 
Transmitted 

phase OOOtcOOtc07C7cO 

The Wa^t/qUM oh "t^e T>PS< sig*-C ii -tk^j o.s -/^/ou;S 



wave 



# — ' 



(b) Let x^. = output of the integrator in the in-phase channel 

Xq = output of the integrator in the quadrature channel 

= one-bit delayed version of Xj 
xq' = one-bit delayed version of xq 
1j = in-phase channel output 



= W 



1q = quadrature channel output 



- XqXq' 



1i + 1q 
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Transmitted 
phase 

(radians) 0 0 0 ti 0 

Polarity of x x + + + - + 

Polarity of Xj + + + - 

Polarity of lj + + - - 

Polarity of Xq - - - + - 

Polarity of Xq - - - + 

Polarity of 1q + + - - 

Polarity of y + + - - 
Reconstructed 

data stream 110 0 



Problem 6.34 



Coherent M-ary PSK requires exact knowledge of the carrier frequency and phase for the receiver 
to be accurately synchronized to the transmitter. When carrier recovery at the receiver is 
impractical, we may use differential encoding based on the phase difference between successive 
symbols at the cost of some degradation in performance. If the incoming data are encoded by a 
phase-shift rather than by absolute phase, the receiver performs detection by comparing the phase 
of one symbol with that of the previous symbol, and the need for a coherent reference is thereby 
eliminated. This procedure is the same as that described for binary DPSK. The exact calculation 
of probability of symbol error for the differential detection of differential M-ary PSK (commonly 
referred to as M-ary DPSK) is much too complicated for M > 2. However, for large values of E/N 0 
and M > 4, the probability of symbol error is approximately given by 



2 M^)> M> - 4 <» 

For coherent M-ary PSK, the corresponding formula for the average probability of symbol error is 
approximately given by 



(a) Comparing the approximate formulas of Eqs. (1) and (2), we see that for M > 4 an M-ary 
DPSK system attains the same probability of symbol error as the corresponding coherent M-ary 
PSK system provided that the transmitted energy per symbol is increased by the following factor: 




k(M) = M>4 

2Sin [iM, 



(b) For example, k(4) = 1.7. That is, differential QPSK (which is noncoherent) is approximately 
2.3 dB poorer in performance than coherent QPSK. 
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Problem 6.35 



(a) For coherent binary PSK, 



e " 2 



For P e to equal 10~\ = 2.64. This yields E^/Ng = 7.0. Hence = 3.5 x 10 -10 . 

The required average carrier power is 0.35 mW. 
(b) For DPSK, 



P e = 2 exp (" iH ' 



4 E 

For P g to equal 10~ , we have j- = 8.5. Hence E fc = 4.3 x 10 -10 . The required average 
power is 0.43 mW. 0 

Problem 6.36 

(a) For a coherent PSK system, the average probability of error is 



P e = ? erfc[/(E b /N Q ) 1 ] 



1 exp[-(E b /N o)l ] 

~ 7 ( 1 ) 



For a DPSK system, we have 

P e = \ exp[- (E b /N Q ) 2 ] (2) 

Let 



348 



Then, we may use Eqs. (1) and (2) to obtain 



/tt /(E b /N Q ) 1 = exp 6 
We are given that 

0 1 

Hence, 



6 = *n[/772T] 
= 1.56 



Therefore, 

E 

10 log 10^ = 10 lo 8in 7 ' 2 = 8 - 5 7 dB 
0 1 



E 

10 log 10^ = 10 lo gio (7 ' 2 + 1 -56) 



0 2 



9.^2 dB 



The separation between the two (E b /N Q ) ratios is therefore 9.42 - 8.57 = 0.85 dB. 
(b) For a coherent PSK system, we have 

P e = \ erfc[/(E b /N Q ) 1 ] 

- ? (3) 
H /(E b /N Q ) 1 

For a QPSK system, we have 



P e = erfc[/7jT70T] 
e b 0 2 

exp[-(E b /N Q ) 2 ] 

/¥ /(E /N )_ 
b 0 2 

Here again, let 

- ♦ < 

0 2 0 1 
Then we may use Eqs. (3) and (4) to obtain 
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exp(-6) 

2 = . <5> 
/1 + 6/(E b /N o)l 

Taking logarithms of both sides: 

-An 2 = - 6 - 0.5 An[1 + 6/CE^Nq).,] 

- - 6 - 0.5 6 



<V M 0>1 



Solving for 6: 

in 2 



6 « 



1 + 0.5/(E b /N Q ) 1 



= 0.65 
Therefore, 

E b 



10 lOg 10^^ 2 = 10 lo 8 10 (7 - 2 + * 65) 



= 8.95 dB. 

The separation between the two (E b /N Q ) ratios is 8.95 - 8.57 = 0.38 dB. 
(c) For a coherent binary FSK system, we have 



P e = \ erfo[/(E b /2» 0 ),] 



«P(- ) 

(6) 



✓(E b /2N Q ) 1 



For a noncoherent binary FSK- system, we have 

E, 

e _ 2 2*N 



P. = i exp(- l(^) ) (7) 



0 2 
Hence, 



plTrJ = expCo) (8) 
0 1 

We are given that (E^N^ = 13. 5. Therefore, 
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= 3.055 
We thus find that 

E, 



10 log 10^ = 10 lo 6 10 (1 3-5) 



0 1 



= 11.3 dB 



10 log 10^ 2 = 10 lo 6 10 (1 3.5 + 3.055) 
= 12.2 dB 

Hence, the separation between the two (E b /N Q ) ratios is 12.2 - 11.3 = 0.9 dB. 
(d) For a coherent binary FSK system, we have 

P e = \ erfc[/(E b /2N 0 )^ ] 

«P(- \^ ) 
1 0 1 



2 /» V (E. /2N n ) 1 
b 0 1 



(9) 



For a MSK system, we have 

P e = 1 erfc[/(E b /2N 0 ) 2 3 (10) 

expC- ^) ] 
0 2 



/? ^E b /2N 0 ) 2 



(10) 



Hence, using Eqs. (9) and (10), we 

in 2 - lin[1 +T1 -^] s l 6 (11) 



Noting that 

6 



( Wl 



« 1 



we may approximate Eq. (11) to obtain 

* n 2 " I C Te4tT-3= I 6 C"> 
b 0 1 

Solving for 6, we obtain 
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6 = 2 &n 2 

1 + w 



2 x 0.693 

= . 1 

1 + 1375 

= 1.29 

We thus find that 

E, 



10 log 10 (jp) = 10 log 1Q (13.5) = 10 x 1.13 = 11.3 dB 



0 1 



10 log 1Q (— ) = 10 log 1Q (13.5 + 1.29) = 11.7 dB 



*0 2 

Therefore, the separation between the two (E b /N Q ) ratios is 11.7 - 11.3 = 0.4 dB. 
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Problem 6 37 




The important point to note here, in comparison to the results plotted in Fig. I is that the error 
performance of the coherent QPSK is slightly degraded with respect to that of coherent PSK and 
coherent MSK. Otherwise, the observations made in Section 8.18 still hold here. 
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Problem 6.38 



The average power for any modulation scheme is 

This can be demonstrated for the three types given by integrating their power spectral 
densities from -» to <*>, 

00 

P = / S(f) df 



= 4 ' [S B (f " V + S B (f + f c )3df 



= 1 / S B (f)df . 

— oo 



The baseband power spectral densities for each of the modulation techniques are: 





PSK 


QPSK 


MSK 


B (f) 


2E b sinc 2 (fT b ) 


HE. sinc 2 (2fTj 
b b 


32E. fcos(2TrfT. )'J 2 
b D ! 

Tt 2 1 6f 2 T 2 - 1 | 
b - 



r°° 1 E 

Since J a smc(ax)dx = 1 , P = — 



is easily derived for PSK and QPSK. For MSK we have 
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16E W « rcos(2irfT. ) \ 2 
P = b j. J b df 

/ _» J 1 6f - 1 J 



16E b " cos 2 (2ttx) j 
—z — / = r dx 

ir T w -» (16x - 1T 



8E. 



j. 1 + cosCUx) dx 



2, .2 



1 



* T b - 16X(X -lt> 



j- cos 0 + cos(Itix) dx 



1 6 it T. -<*> 
b 



(x 2 - -i) 2 
K * 16 ; 



From integral tables, (see Appendix Alffa^ 



/ c ° 3 < ax > d * = _^ [aln(ab) - abcos(ab)] 
0 (b d - x 2 ) 2 4b 3 



For a = 


0, the integral is 0. 
4ir, b = ^, we have 




For a = 




P 


E co £ 
b j cos(ax) dK b 

16tt 2 T k -» (b 2 - x 2 ) 2 T b 
b 




For the 


three schemes, the values of S(f ) 

c 


are as follows: 




PSK QPSK 


MSK 


SCf ) 
c 


h 

2 a b 


8E b 
2 

ir 



Hence, the noise equivalent bandwidth for each technique is as follows: 
i PSK QPSK MSK 



Bandwidth 



1 



1 

2T\ 



0.62 
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Problem 6.39 



(a) Table 1, presented below, describes the differential quadrant coding for the V.32 modem of 
Fig. 6.48a in the textbook, which may operate with nonredundant coding at 9,600 b/s. The 
entries in the table correspond to the following: 

Present inputs: Q\^ n 
Previous outputs: I\,n-ih,n-\ 
Present outputs: 7 lw I 2 ^ n 



Table 1 





Input 


Previous output 


Present output 




dibit 


dibit 


dibit 


Ql,n 


Ql,n 


h,n-l 


l 2,n-\ 


h,n 


l 2,n 


0 




1 


0 


0 


0 


0 


0 




1 


0 


1 


0 


1 


0 




1 


1 


0 


1 


0 


0 




1 


1 


1 


1 


1 


0 




0 


0 


0 


0 


1 


0 




0 


0 


1 


1 


1 


0 




0 


1 


0 


0 


0 


0 




0 


1 


1 


1 


0 






0 


0 


0 


1 


1 






0 


0 


1 


1 


0 






0 


1 


0 


0 


1 






0 


1 


1 


0 


0 






1 


0 


0 


1 


0 






1 


0 


1 


0 


0 






1 


1 


0 


1 


1 






1 


1 


1 


0 


1 
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(b) Table 2, presented below, describes the mapping from the four bits l\ A .\h,n-\^ Q3,nQ4,n to me 
output coordinates of the V.32 modem. 



Table 2 



Present output 


Present input 


Output 


dibit 


dibit 


coordinates 




l 2,n 


Q3,n 


Q4,n 


Ax 

9i 


<l>2 


0 


1 


0 


0 


1 


-1 


0 


1 


0 


1 


1 


-3 


0 


1 


1 


0 


3 


-1 


0 


1 


1 


1 


3 


-3 


0 


0 


0 


0 


-1 


-1 


0 


0 


0 


l 


-3 


-1 


0 


0 


1 


0 


-1 


-3 


0 


0 


1 


l 


-3 


-3 




0 


0 


0 


-1 


1 




0 


0 


1 


-1 


3 




0 


1 


0 


-3 


1 




0 


1 


1 


-3 


3 




1 


0 


0 


1 


1 




1 


0 


1 


3 


1 




1 


1 


0 


1 


3 




1 


1 


1 


3 


3 



>• 4th quadrant 



> 3rd quadrant 



y 2nd quadrant 



1st quadrant 



(b) We are given the current input quadbit: 
and the previous output dibit: 



h,n-lhn-X = 01 



From Table 1, we find that the resulting present output dibit is 



Hence, using this result, together with the given input dibit Q 3>n Q 4yn = 01 in Table 2, we find 
that the coordinates of the modem output are as follows: 
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= 3, and (|> 2 = 1 

We may check this result by consulting Table 6.10 and Pig 6.49 rf ^ ^ 
nj = 00 we find from Table 6.10 that the modem experiences a phase change of 90 . 
With L , h , = 01, we find from Fig. 6.49 that the modem was previously residing m the 

fourth q^t. Hence/with a rotation of 90° in the ^ff*™^f%^ ^ 
moves into the first quadrant. With Q Xn Q A , n = 01, we readily find from Fig. 6.49 that 

<$ ?l = 3, and (j) 2 = 1 

which is exactly the same as the result deduced from Tables 1 and 2 of the solutions manual. 
For another example, suppose we are given 

Ql,nQ2,nQl,nQA,n= 1011 

and 

Then, from Table 1, we find that 

Next from Table 2, we find that the output coordinates are = -3 and * 2 = -3. Confirmation that 
These llts are in perfect accord with the calculations based on Table 6.10 and Figure 6.49 is left 
as an exercise for the reader. 

Problem 6.40 

(a) The average signal-to-noise ratio is defined by 

(SNR) av = — 
a 

where P av is the average transmitted power, and is the channel noise variance. The transmitted 

signal is defined by 

s .(t) = a k cos(2nf c t) - b k sin(2nf c t), 0<t<T 
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where (a b , b b ) is the kth symbol of the QAM signal, and T is the symbol duration. The power 
spectrum of s k (f) has the following graphical form: 



Main lobe 




Fig. 1 

On the basis of this diagram, we may use the null-to-null bandwidth of the power spectrum in Fig. 
1 as the channel bandwidth: 

B = \ or T = - 
T B 

The average transmitted power is 



p =L E - BE ™ 

av y ay 2 



(2) 



where E av is the average signal energy per symbol. 

To calculate the noise variance a 2 , reft > the following figure: 



P jr 
spectrum 
of noise 



Nq/2 



-fc 
B - 



Fig. 2 



f 



359 



The noise variance is therefore 



o = N 0 B 



(3) 



Hence, substituting Eqs. (2) and (3) into (1): 



(SNR) av = 



BE/2 



NqB 



= 

2{N t 



Expressing the SNR in decibels, we may thus write 



101og 10 (SNR) av = -3 + 101og f^|,dB 



Given the value 101og 10 , (EJN 0 ) = 20 dB or EJN Q = 100, we thus have 
101og 1() (SNR) av = 17 dB 



(b) With M = 16, the average probability of symbol error is 



f 



V 



3E„ 



\2(M-\)N Q 



= 1.16 x 10 
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Problem 6.41 



We are given the following set of passband basis functions: 



{<KOcos(27c/„0, Ht)sm(2nf n t)} N n= 
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where /„ = -, n = 1,2, —,N 



and <|)(0 = /|sincf|;,, 



Property 1 



f (Ht)cos(2n f n t))l>((t)sm(2nf n t))dt = 0 for all 



(1) 



To prove this property, we use the following relation from Fourier transform theory: 



f° Mt)cos(2nf n t))(ty(t)&m(2nf n t))dt = 0 for all n 



(2) 



where g,-(r) ^ G ( (/) for / = 1,2, and the asterisk denotes complex conjugation. For the problem at 
hand, we have 



?i(0 



= J slnc ((j 



cos(27t/ n 0 



?2(0 = J| sinc ((^) sm ( 27l /„0 



The Fourier transform of the sine function is 



sincl - 



Trect(JT) 



where 



rect(/T) = 



1 1 

1 for ~ < f < ± 
2T J 2T 



0 



otherwise 



Hence, 



= J|[rect((/-/ w )r) + rect((/ + /„)D] 
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G 2 (f) = ^[rect((/-/ B )r) - rect((/ + /jr)] 

Let /j denote the integral on the left-hand side of Eq. (1). We may then use Eq. (2) to write 

7l = ; (£Ul [reCt2((/_/ " )r) ~ ™ ct2 ((f + fn)T)W 
where the integrand is depicted as follows: 



(3) 









1.0 












1 




Fig. 1 


■fn 

< — \IT ► 


0 


— i — 

« — l/T > 





f 



From Fig. 1 we immediately see that the areas under the two rectangular functions are exactly 
equal. Hence, Eq. (3) is zero, thereby proving Property 1 for any n. 

Property 2 



V2 Aj2 J 2 for k*n 



Let 7 2 denote the integral in Eq. (4). When k = n, we have 



(4) 



1 r°° 2 
= ^ dt 



dt 
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Using Rayleigh's energy theorem, namely, 

r g \t)dt = r \G(ffdf 

» -oo J _oo 

we may write 
1 r°° . 2ft 



f 2 t \ 2 

j sine \j\dt= sine (k)dX, X = t/T 
= f rect 2 (/)J/ 

» — oo 

= 1 

which proves Property 2 for k = n. 
To prove Property 2 for k * n , let 

8 lit) = Ht)e n 



2 . (t\ n*f n t 

= | f sm C - e 



jlK fir' 

g 2 (t) = W)e 



Then applying the following relation from Fourier transform theory, 
f 8i(t)8* 2 (t) dt = f G,{f)G* 2 {f) df 

* — oo * — oo 

we may rewrite the integral I 2 of Eq. (4) as 
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h = 



A J^fn'- 

sine I -je 



A ^/n'" 

sine | -Je 



df 



Since /„ = n/Tby definition, we may depict the two Fourier transforms constituting the integrand 
of I 2 as shown in Fig. 2 for the worst possible case of k = n+l: 



0 



Fig. 2 



f — 2L f — "+1 
Jn~ T T 



-f 



From Fig. 2 we immediately see that the Fourier transforms of §{t)e and §(t)e will 
never overlap for k ^ n . Hence, the integral I 2 is zero, proving the rest of Property 2. 

Property 3 

r;'27t/„r j2nf.t * 

(<K0*M0)e (H0*h(t)e k ) dt = 0 for k*n 
— oo 

where the star denotes convolution. From the convolution theorem, we have 
F[W)*h(t)] = <t>(f)H(f) 

where <D(/) = F[(|)(0] and = F[h(t)] . For jt ?t n , the picture portrayed in Fig. 2 remains 
equally valid except for the fact that the basic rectangular spectrum is now replaced by that 
rectangular spectrum multiplied by the frequency response H(f). This multiplication does not 
affect the nonoverlapping nature of the two spectra representing 

(ty(t)*h(t))e and (<|)(0*M0 e ) for & # « , hence, proving Property 3. 
Problem 6.42 

Step 1 - Set k = 0 and the initial noise-to-signal ratio NSR(k) = 0. Sort the subchannels used in 
ascending order (i.e., from the smallest to largest ones). 

Step 2 - Update the number of subchannels used by setting k = k+l. 
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Step 3 - Compute NSR(k+l) = NSR(k) + 



2 

8k 



Step 4 - Set X(k) = j[P k + TNSR(k)] 

K 



Step 5 - If P k = X(k)-T 



2 



<0 



then compute P { - X(k - 1 ) - P 



f 2 \ 



and 



B, = log. 



1 + _LJ. 

I To;) 



for / = 1, 2, k-l 
Otherwise, go to step 2. 
For notations, refer to Section 6.12. 

(The algorithm presented here is adapted from T. Starr, J.M. Cioffi, and P.J. Silverman (1999);' 
see the bibliography.) 

Problem 6.43 



(a) P l + P 2 + P 3 = P 



P x -K = -r^ = -To 2 



(1) 
(2) 



2 2 

2 u 



(3) 



Si 
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2 2 

Adding Eqs. (2), (3) and (4), and then using Eq. (1): 
3K = p + Ta 2 (l + j- + j- 

Solving for K, we thus have 



v p r<5 2 (, i i 

K = - + -^-| 1 + - + - 



With this value of K at hand, we next solve for P h P 2 , and P 3 , obtaining 



jy p raV. 1 i 

p > = 3 + — l 1 + r 2 -/; 



D P raV 1 2 

P3 = 3 + -rl 1 + /;-r 2 . 



10 in . 
T + 3 2 + 3 



= ^(10 + 2.5) 



12.5 



Po = 



3 3 V 

11 

3 



3) 
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Problem 6.44 



(a) Using matrix notation, the channel output vector is defined by 



x = Ha + w 



where H is the channel matrix, a is the transmitted signal vector, and w is the channel noise 
vector. Applying the singular value decomposition to H, we may write 



H = U AO V+ 



where t denotes Hermitian transposition. The vector-coding receiver uses a bank of discrete 

matched filters defined as the rows of orthonormal matrix IT . Thus passing x through this 
bank of matched filters, we get 

X = U + x 

= U+(Ha + w) 



= Ut U 



AO 



Vta + 



w 



(1) 



Defining 
A = V+a 



and recognizing that U'U = I (identity matrix), Eq. (1) reduces to 

X = AA + W 

where 



(2) 
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W = Ut w 



Each element of the vector X in Eq. (2) represents an independent channel, and so we write 

X n = K\ + W n , n= 1,2, -,N 

(b) In the multichannel transmission model, the channel capacity of the entire system in bits per 
transmission is given by 



N+v 



N+v 



h 2 log 2 



1 + 



1, 

2 l0 §2 



- N 

n 

-n=l 



1 + 



l/(n + v) 



(3) 



where v is the length of the channel impulse response. We may also express the R as follows: 



R = ilog (l + i(SNR) 



vector coding 



(4) 



Hence, combining (3) and (4): 



( N ( 

r + (SNR) vector coding = r Y\ 

V n =lV 



1 



p X\(N+v) 

r<s\>> 



(SNR) vector codi - r 



1_ 

( N ( p X\(N+v) 



n 

V«=l 



V 



To 



-r 



(c) As the block length goes to infinity, we may ignore v, in which case the channel matrix H 
becomes nearly an NxN matrix. Therefore, H may be decomposed as 



H 



H = Q AQ 
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where Q is an orthonormal matrix, and A is a diagonal matrix of eigenvalues (i.e., singular 
values). Correspondingly, the singular values approach the magnitude of the Fourier 
transform. 

Even though a vector coding receiver and discrete multitone receiver converge to the same 
performance, they are not the same: 

• The subchannel gains are the same in both cases, but in vector modulation all subchannels 
have zero phase, while in DMT the subchannels have arbitrary phase angles. 

• Unlike DMT, a vector-coding system does not require the use of a cyclic prefix. 

• The computational complexity of the vector-coding multichannel system is much greater 
than that of its DMT counterpart. 

Problem 6.45 t 





Impulse noise 
(produced in a subscriber 
loop plant) 


Narrowband interference 
(picked up from a nearby 
AM radio station) 


Discrete multitone 
(DMT) system 


The DMT receiver spreads the 
energy of an impulse over many 
subchannels, thereby reducing its 
degrading effect 


Due to the sinc(*) spectral 
characteristic of each subchannel, 
the DMT receiver tends to pass the 
interference into many 
subchannels. Since the signal 
power in each subchannel is only 
1/N of the total signal power, 
susceptibility of the receiver to 
narrowband interference can be 
acute. 


Carrierless 
amplitude/phase 
(CAP) modulation 
system 


The CAP system relies on the use 
of a single carrier. It is therefore 
susceptible to impulse noise. (This 
problem may be mitigated by the 
use of a powerful code such as the 
Reed-Solomon code.) 


The use of an adaptive equalizer is 
based on the mean-square error 
criterion. The effect of narrowband 
interference can therefore be 
reduced by creating a notch in the 
receiver performance at the 
frequency of the interferes 



'The comparative points made in the table presented herein are based on Saltzberg (1998); see the 
bibliography 



Problem 6.46 

In M-ary FSK, the transmitted signal is defined by 
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(1) 



where E is the symbol energy, T is the symbol period, and the carrier frequency f c = nJ2T for 

some fixed integer n c . The signals s t (t) for / = 1, 2, --, M constitute an orthogonal set over the 
interval 0 < t < T, as shown by 



js^sji^dt = 0 for j 



(2) 



Each frequency in Eq. (1) (i.e., specified value of integer i) is modulated with binary data. The net 
result is a set of parallel carriers, each of which contains a certain portion of the incoming user's 
data. What we have just described is a form of orthogonal frequency-division multiplexing 
(OFDM). 

Problem 6.47 

(a) The M-ary PSK signal is given by 



y(t) = Jy-cos[2nf c t + ^(i-l)\, i = 1,2,-,M 



(1) 



The output of the Mth power-law device is the Mh power of the input signal y(t): 

M 



z(t) = \^\ cos M (2nf c t + ^(i-l) 



M 



(2) 



The signal z{t) generates a frequency component at Mf c , which can be used to drive a phase- 
locked loop tuned to Mf c . Specifically, expanding Eq. (2), we get 



z{t) = 



M 



J_ _1_ 

2 M + 2 M- 



(M/2) 


( M 






i X 


M 


k 


cos 


k=\ 


U 


) 





271(2*)/^+ (2*)|5(;-1)" 



M 

2E\ 2 \ 1 (M\ 

— Jcos[27tM/ c f + 27i(/-l)] 



T 



0 
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+ 



1 



2n(M-l)fj + 



(Af-l)27C(f- I)' 
M 



+ 



+ 



1 ( M \ (a * 471,. 1N 
cos 471 f ? + — (l - 1) 



+ (m/2) 2 M J 

Therefore according to the first term of this series expansion, the output of the Mth power law 
device contains a tone of frequency Mf c , where f c is the original carrier frequency. 

(b) The phase-locked loop is set to a frequency equal to Mf c . The phase-locked loop acts as a 
narrow-band filter, thereby passing the sinusoidal component of frequency Mf c and rejecting 
the other components. 

(c) Consider, for example, the simple case of binary PSK. Since a squaring loop contains a 
squaring device at its input end, it is clear that changing the sign of the input signal leaves the 
sign of the recovered carrier unaltered. In other words, the squaring loop with M=2 exhibits a 
180° phase ambiguity. Generalizing this result, we may say that .Mth power loop for M-ary 
PSK exhibits M phase ambiguities in the interval [0,27t]. 

One method of resolving the phase ambiguity problem is to exploit differential encoding. 
Specifically, the incoming data sequence is first differentially encoded before modulation, 
resulting in a small degradation in noise performance. This method is called the coherent 
detection of differentially encoded M-ary PSK. As such, this method of modulation is 
different from the M-ary DPSK considered in Problem 6.34. For the special case of coherent 
detection of differentially encoded binary PSK, the average probability of symbol error is 
given by 



erfc 


f 


F ) 


1 , 2 
- ^rf c 


/ 











(3) 



In the region where CE//V 0 ) »1, the second term on the right-hand side of Eq. (3) has a 
negligible effect; hence, this modulation scheme has an average probability of symbol error 
practically the same as that for coherent QPSK or MSK. For the coherent detection of 
differentially encoded QPSK, the average probability of symbol error is given by 



P e = 2erfc 




-2erfc 




+ erfc' 
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For large E^Nq, this average probability of symbol error is approximately twice that of 
coherent QPSK. 

Problem 6.48 

(a) Assuming that the input data sequence x[n] is measured in volts, and recognizing that the 
symbol a n is dimensionless, then from Eq. (6.271) we find that the error signal e[n] is also 

measured in volts. Then, with the phase estimate Q[n] measured in radians, it follows that the 
step-size parameter y for carrier recovery in Eq. (6.272) is measured in radians/volts. 

(b) From Eq. (6.282) defining the error signal in terms of the input data sequence, we see that the 
error signal is measured in volts squared. Hence, with c[n] responsible for timing recovery, 
measured in seconds, it follows that the step-size parameter y for timing recovery in Eq. 
(6.286) is measured in seconds/volts 2 . 

Problem 6.49 

(a) The complex envelope of the received waveform is given by 

~r(t) = ~s(t) + w(t) (1) 

l 0 -i 

where s(t) = e j{2nvt + 6) j a k g(t-kT-x) 

k=0 

and w(t) is the channel noise. The parameter v represents the frequency offset, 0 is the carrier 
phase we want to estimate, t is the timing error, {a k } is the sequence of information symbols, 
T is the symbol period, and g(t) is the signaling pulse shape. 

The likelihood function L(r|0) is given by 




L Q -l 

where ~s(t) = e j{2KVt + Q) ^ a k g(t - kT -t) 

k=0 

Since \s(t)\ is independent of the carrier phase Q , the log-likelihood function of 0 is given by 
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/(0)=log(L(r|0)) = Re 



J r(t)s*(t)dt 



.V 1 



where J" ~r{t)~s* {t)dt = e jQ ^ a*jc(fc) 



where jc(fc) represents the sample taken at time t = kT + % in the formula for convolution: 



Therefore, 



-V 1 



1(B) = Rele j6 ^a k x(k)i 

t k=0 J 

The maximum of /(9) , i.e., maximum likelihood estimation of 0> is achieved for 



,l 0 -i 



6 = ar § X a k x ( k ) 
t k=0 



(b) Hence, from Eq. (4) we deduce the following system for estimating the phase 0 : 



r(0 



-MX) 



g(-t) 



x(k) 



MX 



arg{-} 



A 

-> 0 



J2nvt 



t=kT+x 



* 

a k 
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Problem 6.51 



Matlab codes 

'/. Problem 6.51 a. CS: Haykin 

'/.effect of a dispersive channel on BPSK signals 
'/. M. Sellathurai 

*/. number of bits and number of samples per bit 

no_of_syms =10; 

no_of _bits=no_of _syms*l ; 

samples_per_bit=16 ; 

*/. generating bits 

Bits=[l 10110100 1]'; 



*/. generating QPSK signals 
[syms]=BPSK_mod(no_of_bits, Bits); 

ts=le-3/16; 
l=length(syms) ; 

'/. baseband signal 

s=zeros(samples_per_bit*(l) ,1) ; 
for k=l:l-l 

for kk=0: (samples_per_bit-l) 

s ( (k-1) *samples_per_bit+kk+l , l)=syms ( 1 ,k) ; 

end 

end 

t=0:ts: (length(s)-l)*ts; 

'/. channel bandwidths 2B=12, filter order 2N=10 
B=6; N=5; 

H12=butter_channel(2*B,N) ; 
TT12=conv(H12, s); 

'/. channel bandwidths 2B=16 filter order 2N=10 
B=8; 

H16=butter_channel(2*B,N) ; 
TT16=conv(H16, s); 
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*/. channel bandwidth 2*B=20 filter order 2N=10 
B=10; 

H20=butter_channel(2*B,N) ; 
TT20=conv(H20, s); 



'/.channel bandwidth 2*B=24 filter order 2N=10 
B=12; 

H24=butter_channel(2*B,N) ; 
TT24=conv(H24, s); 

'/. channel bandwidth 2*B=30 filter order 2N=10 
B=15; 

H30=butter_channel(2*B,N) ; 
TT30=conv(H30, s); 

'/, prints 

subplot (2, 3,1) 
hold on 
for k=l:10 

plot(k, [syms(k)] ' , 'o') 

line([k, k] , [0 syms(k)]) 
end 

xlabel('Bit period'); 
title( 'Transmitted bits'); 
hold off 



subplot (2, 3 ,2) 

[ m start] =max (real (H12)) ; 

hold on 

plot(t,s, ' — '); 

plot (t .real (TT12(start:160+start-l))) ; 
xlabeK'time (s) '); 
title ('Baseband BPSK, BW=12kHz'); 
hold off 

subplot (2, 3 ,3) 

[m start] =max(real(H16)) ; 

hold on 

plot(t,s,' — '); 

plot(t,real(TT16(start:160+start-l))); 
xlabeK'time (s) '); 
title(' Baseband BPSK, BW=16kHz ' ) ; 
hold off 
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subplot (2, 3, 4) • 

[m start] =max(real(H20) ) ; 

hold on 

plot(t,s, >--'); 

plot (t, real (TT20(start : 160+start-l) ) ) ; 
xlabeK'time (s)'); 
title( 'Baseband BPSK, BW=20kHz'); 
hold off 

subplot (2, 3, 5) 

[m start] =max (real (H24) ) ; 

hold on 

plot(t,B, ' — '); 

plot(t,real(TT24(start: 160+start-l))) ; 
xlabeK'time (s)'); 
title ('Baseband BPSK, BW=24kHz'); 
hold off 

subplot (2, 3, 6) 

Cm start] =max(real(H30) ) ; 

hold on 

plot(t,s,' — '); 

plot (t , real (TT30(start : 160+start-l) ) ) ; 
xlabeK'time (s)'); 
title( 'Baseband BPSK, BW=30kHz'); 
hold off 
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'/. Problem 6.51 b. CS: Haykin 

'/.effect of a dispersive channel on QPSK signals 
'/. M. Sellathurai 

'/, number of bits and number of samples per bit 

no_of_syms =5; 

no_of _bits=no_of _syms*2 ; 

samples_per_bit=16 ; 

*/. generating bits 
'/.Bits=round(rand(no_of_bits, 1)) ; 
Bits=[i 10 110 1001]'; 

'/. generating QPSK signals 
[syms]=QPSK_mod(no_of_bits, Bits) ; 

l=length(syms) ; 

'/, baseband signal 
s=zeros(samples_per_bit*(l-l) , 1) ; 
for k=l:l-l 

for kk=0: (samples_per_bit-l) 

s ( (k-1 ) *samples_per_bit+kk+l , 1 )=syms ( 1 ,k) ; 

end 

end 

t=0:ts: (length(s)-l)*ts; 

'/. channel bandwidths 2B=12, filter order 2N=10 
B=6; N=5; 

H12=butter_channel(2*B,N) ; 
TT12=conv(H12, s); 

7. channel bandwidths 2B=16 filter order 2N=10 
B=8; 

H16=butter_channel(2*B,N) ; 
TT16=conv(H16, s); 



'/. channel bandwidth 2*B=20 filter order 2N=10 
B=10; 

H20=butter_channel(2*B,N) ; 
TT20=conv(H20, s); 
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'/.channel bandwidth 2*B=24 filter order 2N=10 
B=12; 

H24=butter_channel(2*B,N) ; 
TT24=conv(H24, s); 

*/. channel bandwidth 2*B=30 filter order 2N=10 
B=15; 

H30=butter_channel(2*B,N) ; 
TT30=conv(H30, s); 

'/. prints 
subplot (2, 3, 1) 
hold on 
for k=l: 10 

plot(k, [2*Bits(k)-l] > , >o') 
line([k, k] , [0 (2*Bits(k)-l)] ) 
end 

xlabel('Bit period'); 
title( 'Transmitted bits'); 
hold off 



subplot (2, 3, 2) 

[ m start]=max(real(H12)) ; 

hold on 

plot(t,s, ' — '); 

plot (t , real (TT12 ( start : 64+st art- 1 ) ) ) ; 
xlabel( 'time (s)'); 
title ('Baseband QPSK, BW=12kHz'); 
hold off 

subplot (2, 3, 3) 

[m start]=max(real(H16)); 

hold on 

plot(t,s, ' — '); 

plot (t , real (TT16 ( start : 64+start- 1 ) ) ) ; 
xlabeK'time (s) '); 
title(' Baseband QPSK, BW=16kHz'); 
hold off 

subplot(2,3,4) 

[m start] =max(real(H20) ) ; 

hold on 

plot(t,s, ' — '); 

plot (t , real (TT20 (start : 64+start-l ) ) ) ; 
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xlabeK'time (s)'); 

title( 'Baseband QPSK, BW=20kHz'); 

hold off 

subplot (2, 3, 5) 

[m start]=max(real(H24)); 

hold on 

plot(t,s, ' — '); 

plot (t , real (TT24 ( start : 64+start- 1 ) ) ) ; 
xlabeK'time (s)'); 
title ('Baseband QPSK, BW=24kHz'); 
hold off 

subplot (2, 3 ,6) 

[m start]=max(real(H30)); 

hold on 

plot (t, a, ' — '); 

plot (t , real (TT30( start : 64+start-l) ) ) ; 
xlabel('time (s)'); 
title ('Baseband QPSK, BW=30kHz'); 
hold off 
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'/. Problem 6.51 c. CS: Haykin 

'/.effect of a dispersive channel on MSK signals 
'/. M. Sellathurai 

'/. number of bits and number of samples per bit 

no_of_syms =5; 

no_of _bits=no_of _syms*2 ; 

samples_per_bit=16 ; 

'/. generating bits 

Bits=[l 1 0 1 1 0 1 0 0 0]'; 

*/. generating QPSK signals 

[s , phase] =MSK_mod(no_of _bits , samples_per_bit , Bits ) ; 

'/. channel bandwidths 2B=12, filter order 2N=10 
B=6; N=5; 

H12=butter_channel(2*B,N) ; 
TT12=conv(H12, s); 

*/. channel bandwidths 2B=16 filter order 2N=10 
B=8; 

H16=butter_channel(2*B,N) ; 
TT16=conv(H16, s); 



'/, channel bandwidth 2*B=20 filter order 2N=10 
B=10; 

H20=butter_channel(2*B,N) ; 
TT20=conv(H20, s); 



'/.channel bandwidth 2*B=24 filter order 2N=10 
B=12; 

H24=butter_channel(2*B,N) ; 
TT24=conv(H24, s); 

'/. channel bandwidth 2*B=30 filter order 2N=10 
B=15; 

H30=butter_channel(2*B,N) ; 
TT30=conv(H30, s); 
ts=le-3/16; 

t=0:ts: (length(s)-l)*ts 
'/. prints 
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subplot (2, 3,1) 
hold on 
for k=l: 10 

plot(k, [2*Bits(k)-l] ' , 'o') 
line([k, k], [0 (2*Bits(k)-l)] ) 
end 

xlabel('Bit period'); 
title( 'Transmitted bits'); 
hold off 



subplot (2, 3, 2) 

[ m start] =max(real(H12) ) ; 

hold on 

plot(t,abs(s) , ' — >); 

plot (t , abs (TT12 (start+5 : 165+start-l ) ) ) ; 
xlabeK'time (s) ') ; 
title('MSK (envelope), BW=12kHz'); 
hold off 

axis([0, 0.01,0.9,1.1 ]) 

subplot (2, 3, 3) 

[m start]=max(real(H16)); 

hold on 

plot(t,abs(s), ' — '); 

plot (t , abs (TT16 (start+5 : 165+start-l ) ) ) ; 
xlabeK'time (s)'); 
title (' MSK (envelope), BW=16kHz'); 
hold off 

axis([0, 0.01,0.9,1.1 ]) 

subplot (2, 3, 4) 

[m start] =max (real (H20)); 
hold on 

plot(t,abs(s), ' — '); 

plot (t , abs (TT20(start+5 : 165+start-l) ) ) ; 
xlabeK'time (s) '); 

title('MSK (envelope), BW=20kHz'); 
hold off 

axis([0, 0.01,0.9,1.1 ]) 

subplot (2, 3, 5) 

[m start]=max(real(H24)); 
hold on 

plot(t,abs(s) ,' — '); 

plot (t , abs (TT24(start+5 : 165+start-l ) ) ) ; 
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xlabeK'time (s)'); 

title('MSK (envelope), BW=24kHz'); 

hold off 

axis([0, 0.01,0.9,1.1 ]) 

subplot (2, 3, 6) 

[m start] =max(real(H30) ) ; 

hold on 

plot(t,abs(s), ' — '); 

plot (t(l: 155) ,abs(TT30(start+5: 160+start-l))) ; 

xlabeK'time (s)'); 

title('MSK (envelope), BW=30kHz'); 

hold off 

axis([0, 0.01,0.9,1.1 ]) 
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'/. Problem 6.51 d. CS: Haykin 

'/.effect of a dispersive channel on GMSK signals 
*/. M. Sellathurai 

'/. number of bits and number of samples per bit 
no_of_syms =5; 
no_of_bits= no_of _syms*2; 
samples_per_bit=16 ; 

'/, generating bits 

Bits=[l 1 0 1 1 0 1 0 0 0]'; 

'/. generating GMSK signals, WTb=0.3 

[s , phase] =GMSK_mod(no_of _bits , samples_per_bit , Bit s ) ; 

'/. channel bandwidths 2B=12, filter order 2N=10 
B=6; N=5; 

H12=butter_channel(2*B,N) ; 
TT12=conv(H12, s); 

*/. channel bandwidths 2B=16 filter order 2N=10 
B=8; 

H16=butter_channel(2*B,N) ; 
TT16=conv(H16, s); 



'/. channel bandwidth 2*B=20 filter order 2N=10 
B=10; 

H20=butter_channel(2*B,N) ; 
TT20=conv(H20, s); 



'/.channel bandwidth 2*B=24 filter order 2N=10 
B=12; 

H24=butter_channel(2*B,N) ; 
TT24=conv(H24, s); 

'/. channel bandwidth 2*B=30 filter order 2N=10 
B=15; 

H30=butter_channel(2*B,N) ; 
TT30=conv(H30, s); 

ts=le-3/16; 

t=0:ts: (length(s)-l)*ts 
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'/. prints 
subplot (2, 3, 1) - 
hold on 
for k=l:10 

plot(k, [2*Bits(k)-i] ' , >o') 
line([k, k] , [0 (2*Bits(k)-l)] ) 
end 

xlabeK'Bit period'); 
title( 'Transmitted bits'); 
hold off 
subplot(2,3,2) 

[ m start] =max (real(H12) ) ; 
hold on 

plot(t,abs(s) , ' — '); 

plot (t , abs (TT12 (start : 160+start-i ) ) ) ; 

xlabeK'time (s)'); 

title('GMSK (envelope), BW=12kHz'); 

hold off 

axis([0, 0.01,0.9,1.1 ]) 

subplot (2, 3, 3) 

[m start] =max(real(H16) ) ; 

hold on 

plot(t,abs(s) , ' — '); 
plot(t,abs(TT16(start: 160+start-l) ) ) ; 
xlabeK'time (s)'); 

title (' GMSK (envelope), BW=16kHz'); 
hold off 

axis([0, 0.01,0.9,1.1 ]) 

subplot (2, 3, 4) 

[m start]=max(real(H20)); 

hold on 

plot(t,abs(s), ' — '); 

plot (t , abs (TT20 (start : 160+start-l ) ) ) ; 

xlabel('time (s)'); 

title('GMSK (envelope), BW=20kHz'); 

hold off 

axis([0, 0.01,0.9,1.1 ]) 

subplot (2, 3, 5) 

Cm start]=max(real(H24) ) ; 

hold on 

plot(t,abs(s) , ' — '); 

plot (t, abs (TT24(start: 160+start-l))) ; 
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xlabeK'time (s)'); 

title('GMSK (envelope), BW=24kHz'); 

hold off 

axis([0, 0.01,0.9,1.1 ]) 

subplot (2, 3, 6) 

[m start]=max(real(H30)); 

hold on 

plot(t,abs(s) , '--'); 

plot (t,abs(TT30( start: 160+start-l) ) ) ; 

xlabeK'time (s)'); 

title('GMSK (envelope), BW=30kHz'); 
hold off 

axis([0, 0.01,0.9,1.1 ]) 
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function [amp]=BPSK_mod(no_of_bits, b) 

'/. used in problem 6.51(a), CS: Haykin 

*/. BPSK modulation 

'/. Mathini Sellathurai 

amp= [] ; 

1=1; 

m=size(b, 1) ; 

for k=l : 1 :m 

if (b(k)==0 ) 

amp(l)= (-1); 

elseif (b(k)==l ) 

amp(l)= 1; 

end 

1=1+1; 

end 
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function [amp]=QPSK_mod(no_of _bits, b) 

'/. used in problem 6.51(b), CS: Haykin 

*/. qPSK modulation 

*/, Mathini Sellathurai 

amp= [] ; 

1=1; 

m=size(b, 1) ; 
for k=l:2:m 

if (b(k)==0 & b(k+l) == 0) 
amp(l)= (-l+i*-l)/sqrt(2) ; 
elseif (b(k)==l & b(k+l) == 0) 
amp(l)= (l-i*l)/sqrt(2); 
elseif (b(k)==l & b(k+l) == 1) 
amp(l)= (l+i*i)/sqrt(2) ; 
else (b(k)==0& b(k+l) == 1) 
amp(l)= (-i+i*l)/sqrt(2) ; 

end 

1=1+1 

end 
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function [amp, phase] =MSK_mod(no_of _bits , samples_per_bit , b) 
*/. used in problem 6.51(c), CS: Haykin 
'/. MSK signal generator 
'/. Mathini Sellathurai 

amp= [] ; 
ini_phase=0; 



for k=l:no_of_bits 
ee=b(k) ; 

for kk=0 : samples_per_bit-l 

'/. NRZ signal generator 
if ee==0 

ee=-i ; 
elseif ee==l 

ee=l ; 

end 

phase((k-l)*samples_per_bit+kk+l)=ini_phase +ee*(pi/(2*samples_per_bit)) ; 
ini_phase=phase((k-l)*samples_per_bit+kk+l) ; 

end 
end 

phase=rem(phase,2*pi) ; 
in=cos (phase) ; 
quad=s in (phase) ; 
amp=in+i*quad; 
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function [amp, phase i]=GMSK_mod(no_of _bits , samples_per_bit , b) 
'/. used in problem 6.51(d), CS: Haykin 
'/. GMSK signal generator 
'/. Mathini Sellathurai 

amp= [] ; 

for k =l:no_of_bits 

'/.Generating NRZ sequence 

if b(k,l)==0 
im_bits(k,l)= -i; 

else 
im_bits(k,l) =1; 

end 
end 

impulse_bits=im_bits ; 

Bits_to_transmit=max (size ( impulse_bits ) ) ; 

BT=0 . 6 ; 

inphase=0; 

data(l,4)=0; 

t=0; 

for i=0:3 

for k=0: (samples_per_bit -1) 

co =GMSK_co(i-2,k+8,samples_per_bit,BT) ; 
qmskcoef (1, i*samples_per_bit+k+l)=co; 

end 
end 

for bitcount=i:Bits_to_transmit 
ini_phase=inphase ; 

ini_phase=rem(ini_phase+data(l,4)*pi/2,2*pi) ; 

data(l, l)=impulse_bits(bitcount, 1) ; 
for i =4:-l:2 
data(l,i)=data(l,i-l) ; 
end 

inphase=ini_phase ; 

for pha_loop=l : samples_per_bit 

phase=inphase; 
for i=0:3 

phase=phase+pi/2*data( 1 , i+1 ) *qmskcoef ( 1 , samples_per_bit*i+pha_loop) ; 
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end 

samples_store(l,t+l)=t ; 
t=t+l; 

phase=rem (phase, 2*pi) ; 

phasel ( 1 , pha_loop+samples_per_bit* (bitcount-1 ) )=phase ; 

rephase(l,pha_loop+samples_per_bit*(bitcount-l))=cos(phase): 

quphase(.l,pha_loop+samples_per_bit*(bitcount-l))=sin(phase); 
end 

end 

amp=rephase+j *quphase ; 
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function [co] = GMSK_co(a, b, samples_per_bit ,bt) 

*/, used in GMSK signal generation Problem 651d, CS: Haykin 

'/. Mathini Sellathurai 

alpha=bt*5 . 336446225 ; 
T=a+b/ samples_per_bit ; 

co=T*erf (T*alpha)+exp(-alpha*alpha*T*T)/(alpha +1.772453855) ; 
co=co-(T-l)*erf ( (T-l)*alpha)-exp(-alpha*alpha*(T-l)* (T-l) )/(alpha*sqrt (pi) ) ; 
co=0.5+0.5*co; 
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function hb=butter_channel(f ,N) 
'/. Used in Problem 6.51 
'/. Butterworth filter of order 2N=10; 
'/. M. Sellathurai 

[B, A]=butter(N, f /64) ; 
[H,w]=freqz(B,A,128, 'whole') ; 
hb=ifft(H) ; 
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Answer to Problem 6.51 




Figure I -. Butterworth baseband filter of order N=5 
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Figure 3 Phase of baseband MSK signal 



398 




time (s) 

Figure ^ I and Q components of baseband MSK signal 
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CHAPTER 7 
Spread-spectrum Modulation 



Problem 7.1 



(a) The PN sequence length is 

N = 2 m - 1 = 2 4 - 1 = 15 

(b) The chip duration is 



T c = _Ls = 0.1 us 
10 7 



(c) The period of the PN sequence is 



T = NT C 



= 15 x 0.1 = 1.5 us 



Problem 7.2 



Shift Shift-register 
number contents 



0 


1000 


1 


0100 


2 


0010 


3 


1001 


4 


1100 


5 


0110 


6 


1011 


7 


0101 


8 


1010 



Modulo-2 Shift-register 



adder output 


output 


0 + 0 = 0 


0 


0 + 0 = 0 


0 


1 + 0 = 1 


0 


0 + 1 = 1 


1 


0 + 0 = 0 


0 


1 + 0 = 1 


0 


1 + 1 = 0 


1 


0 + 1 = 1 


1 



9 1101 1 + 0 = 1 0 

10 1110 0 + 1 = 1 1 

11 1111 1 + 0 = 1 0 

12 0111 1 + 1 = 0 1 

13 0011 1 + 1 = 0 1 

14 0001 1 + 1 = 0 1 

15 1000 0 + 1 = 1 1 



The output sequence is therefore 

ii o oo loo lm oim ooo i 

one period 

Problem 7.3 

(a) From both Table 7-2a and Table 7.2b we note the following: 

Balance property: 

Number of Is in one period = 16 
Number of 0s in one period = 15 

Hence, the number of Is exceeds the number of 0s by one. 

(b) Run property: 

In both Tables 7.2a and 7 2b, we count a total of 8 runs of Is and a total of 8 runs of 0s. Moreover, 
we note the following: 

Runs of length 1 : 4 
Runs of length 2 : 2 
Runs of length 3 : 1 



(c) Autocorrelation function: 
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Problem 7.4 




Table 1 



Feedback State of Feedback- Output 
symbol shift register symbol 
10 0 0 0 0 



1 


110 0 0 0 


0 


1 


1110 0 0 


0 


1 


11110 1 


0 


1 


111110 


0 


1 


111111 


0 


u 


0 11111 

\J X X X X X 


1 

J. 


1 


10 1111 


1 


0 


0 10 111 


1 


1 


10 10 11 


1 


0 


0 10 10 1 


1 


1 


10 10 10 


1 


1 


110 10 1 


0 


0 


0 110 10 


1 


0 


0 0 110 1 


0 


1 


10 0 110 


1 


1 


110 0 11 


0 


0 


0 110 0 1 


1 


1 


10 110 0 


1 


1 


110 110 


0 


1 


1110 11 


0 


0 


0 1110 1 


1 


1 


10 1110 


1 



Table 1 continued 



Feedback State of feedback- Output 
symbol » shift register symbol 



1 


110 111 


0 


0 


0 110 11 


1 


1 


10 110 1 


1 


0 


0 10 110 


1 


0 


0 0 10 11 


0 


1 


10 0 10 1 


1 


0 


0 10 0 10 


1 


0 


0 0 10 0 1 


0 


1 


10 0 10 0 


1 


1 


110 0 10 


0 


1 


1110 0 1 


0 


0 


0 1 1 1 0 c 


1 


0 


0 0 1110 


0 


0 


0 0 0 111 


0 


1 


10 0 0 11 


1 


0 


0 1000 1 


1 


1 


10 1000 


1 


1 


110 10 0 


o 


1 


1110 10 


0 


1 


11110 1 


0 


0 


0 11110 


1 


0 


0 0 1111 


0 


1 


10 0 111 


1 


0 


0 100 11 


1 


1 


10 10 0 1 


1 


0 


0 10 10 0 


1 


0 


0 0 10 10 


1 
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Table 1 continued 



Feedback State of feedback- Output 
symbol shift register symbol 



0 


0 0 0 10 1 


0 


1 


10 0 0 10 


1 


1 


11 0 0 0 1 


0 


0 


0 110 0 0 


1 


0 


0 0 110 0 


0 


0 


0 0 0 110 


0 


0 


0 0 0 0 11 


0 


1 


10000 1 


0 


0 


0 10000 


1 


0 


0 0 10 0 0 


0 


0 


0 0 0 10 0 


0 


0 


0 0 0 0 10 


0 


0 


0 0 0 0 0 1 


0 


1 


10 0 0 0 0 


1 
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4-p~ 1 I * 2 



+— 3 



* 4 



-» 5 



Table 2 



Feedback State of feedback- Output 
symbol shift register symbol 

10 0 0 0 0 



1 


110 0 0 0 


0 


0 


0 110 0 0 


0 


1 


10 110 0 


0 


1 


110 110 


0 


1 


1110 11 


0 


0 


0 1110 1 


1 


0 


0 0 1110 


1 


1 


10 0 111 


0 


1 


110 0 11 


1 


0 


0 110 0 1 


1 


0 


0 0 110 0 


1 


0 


0 0 0 110 


0 


1 


10 0 0 11 


0 


1 


110 0 0 1 


1 


1 


1110 0 0 


1 


0 


0 1110 0 


0 


1 


10 1110 


0 


0 


0 10 111 


0 


1 


10 10 11 


1 


1 


110 10 1 


1 


1 


1110 10 


1 


1 


11110 1 


0 


1 


111110 


1 


1 


1 1 1 1 1 1 


0 


0 


0 11111 


1 
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Table 2 continued 



Feedback State of feedback- Output 
symbol shift register symbol 



1 


10 1111 


1 


1 


110 111 


1 


0 


0 110 11 


1 


1 


10 110 1 


1 


0 


0 10 110 


1 


0 


0 0 10 11 


0 


0 


0 0 0 10 1 


1 


1 


1000 10 


1 


0 


0 10 0 0 1 


0 


0 


0 0 1 0 0 0 


1 


0 


0 0 0 10 0 


0 


0 


0 0 0 0 10 


0 


1 


10000 1 


0 


0 


0 10000 


1 


1 


10 10 0 0 


0 


1 


110 10 0 


o 


0 


0 110 10 


0 


0 


0 0 110 1 


0 


1 


100 110 


1 


0 


0 10 0 11 


0 


1 


10 1001 


1 


0 


0 10 10 0 


1 


1 


10 10 10 


0 


0 


0 10 10 1 


0 


0 


0 0 10 10 


1 
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Table 3 



Feedback State of feed-back Output 
symbol shift register symbol 
1 00000 



0 


0 10000 


0 


1 


10 10 0 0 


0 


1 


110 100 


0 


1 


1110 10 


0 


1 


11110 1 


0 


1 


111110 


1 


1 

J. 


111111 

J. X J. X J. J- 


o 


0 


0 11111 


1 


0 


0 0 1111 


1 


1 


10 0 111 


1 


0 


0 10 0 11 


1 


1 


10 1001 


1 


0 


0 10 10 0 


1 


1 


10 10 10 


0 


0 


0 10 10 1 


0 


0 


0 0 10 10 


1 


0 


0 0 0 10 1 


0 


1 


10 0 0 10 


1 


1 


11000 1 


0 


0 


0 11000 


1 


0 


0 0 110 0 


0 
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Table 3 continued 



Feedback 


State of feedback- 


Output 


symbol 


shift register 


symbol 


1 


10 0 110 


0 


1 


110 0 11 


0 


1 


1110 0 1 


1 


1 


11110 0 


1 


0 


0 11110 


0 


1 


10 1111 


0 


1 


110 111 


1 


1 


1110 11 


1 


0 


0 1110 1 


1 


1 


10 1110 


1 


0 


0 10 111 


0 


1 


10 10 11 


1 


1 


110 10 1 


1 


0 


0 110 10 


1 


1 


10 110 1 


0 


0 


0 10 110 


1 


0 


0 0 10 11 


0 


1 


10 0 10 1 

J- V/ \J J- \J -J. 


1 

J. 


1 


110 0 10 


1 


0 


0 11001 


0 


1 


10 110 0 


1 


1 


110 110 


0 


0 


0 110 11 


0 


0 


0 0 110 1 


1 


0 


0 0 0 110 


1 
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Table 3 continued 



Feedback State of feedback- Output 
symbol shift register symbol 



1 


10 0 0 11 


0 


o 


0 10 0 0 1 


1 


o 


0 0 10 0 0 


1 


1 


10 0 10 0 


0 


0 


0 10 0 10 


0 


0 


0 0 10 0 1 


0 


0 


0 0 0 10 0 


1 


u 


ft ft n n i ft 

U U U U 1 u 


ft 


1 


10000 1 


0 


1 


110 0 0 0 


1 


1 


111000 


0 


0 


0 1110 0 


0 


0 


0 0 1110 


0 


0 


000 111 


0 


0 


0 0 0 0 11 


1 


0 


0 0 0 0 0 1 


1 


1 


10 0 0 0 0 


1 
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Table 3 continued 

Feedback State of feedback- Output 
symbol shift register symbol 



1 


10 0 10 1 


0 


0 


0 10 0 10 


1 


0 


0 0 10 0 1 


0 


1 


10 0 10 0 

JL \J \J -A. \J \J 


1 


1 


110 0 10 


0 


1 


1110 0 1 


0 


1 


111100 


1 


0 


0 11110 


0 


0 


0 0 1111 


0 


0 


0 0 0 111 


1 


0 


0 0 0 0 1 1 


1 


0 


0 0 0 0 0 1 


1 


1 


10 0 0 0 0 


1 
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Problem 7.5 



State of feedback- Output 
shift register symbol 

Initial sfde 1 0 0 0 0 



0 1000 


0 


0 0 10 0 


0 


000 10 


0 


0 0 0 0 1 


0 


1110 1 


1 


10 0 11 


1 


10 10 0 


1 


0 10 10 


0 


0 0 10 1 


0 


mil 


1 


10 0 10 


1 


0 10 0 1 


0 


110 0 1 


1 


1 0 0 0 1 


1 


10 10 1 


1 


10 111 


1 


10 110 


1 


0 10 11 


0 


110 0 0 


1 


0 110 0 


0 


0 0 110 


0 


0 0 0 1 1 


0 


11100 


1 


OHIO 


0 


0 0 111 


0 


11110 


1 


0 1111 


0 


110 10 


1 


0 110 1 


0 


110 11 


1 


1 0 0 0 0 


1 



The 31-element code generated by the scheme shown in Fig. P9.2 is exactly the same as that 
described in Table 9.2b. Note, however, the code described in Tab.e 9.2b appears in reversed 
order to that described in the above table; this reversal is clearly of a trivial nature. 
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Problem 7.6 



(a) The modulo-2 sum of b(t) and c(t), on a pulse-by-pulse basis, is as follows 



c(t) 



0 
1 



0 1 

1 0 



(b) If symbol 0 is represented by a sinusoid of zero phase shift, and symbol 1 is 
represented by a sinusoid of 180° phase shift, the output of the modulo-2 adder takes on 
the same form as that described in Table 7 .3 of the text. 

Problem 7.7 



j(t) = fiJ cos(27tf c t + 6) 



The basis functions are 



4>k(t) = }\ T 



cos(27tf c t) , 



0, 



kT c < t < (k + l)T c 
otherwise 



<|> (t) = 1 
k 



_f_ sin(2rcf c t), 
0, 



IN T c 



kT c < t < (k + l)T c 
otherwise 



Hence, we may express the jamming signal j(t) as 
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N-l 

j(t) = /JT C cos9 £ <|> k (t) 

k=0 



N-l 

E 

k=0 

N-l - 

- ^JT C sine £ <|> (t) 
k=0 k 



Problem 7.8 

The processing gain is 



T c 1/T b 

The spread bandwidth of the transmitted signal is proportional to 1/T C . The despread 
bandwidth of the received signal is proportional to 1/T b MeT>ae> 



Processing gain = s P read bandwidth of transmitted signal 

despread bandwidth of received signal 



Problem 7.9 

m = 19 

N = 2 m -l = 2 19 -1 - 2 19 



The processing gain is 



101og 10 N - 101og 10 2 

= 190 x 0.3 
= 57 dB 



19 
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Problem 7.10 

(a) Processing gain = 101og 10 (2 m -l) = 101og 10 (2 19 -l) = 57 

(b) Antijam margin = (Processing gain) - 101og 10 
The probability of error is 



f \ 
\ j 



P p = — erfc 
e 2 



1*0 



With P e = 10" 5 , we have Ej/Nq = 9. Hence, 

Antijam margin = 57 - 101og 10 9 = 57 - 9.5 

= 47.5 dB 
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Problem 7. 1.1 

The DS/QPSK signal modulator is given below: 

Cj(t) 



bit) 
(binary 
data) 



NRZ 
encoder 



s{t) 



■KX 



Baseband 
filter 



-jr- cos(2ti/ c ?) 
' c 

*&) — 



) — 


Baseband 






filter 


K 



QPSK 
signal 



- sin(27t/ c f) 



The DS/QPSK modulated signal is 
x(t) = ±^c / (/)cos(27i/ c O±^c G (Osin(27t/ c O 

where C/ (f) = {c 0>I (t), c XJ {t), c N ._ hI (t)} and 
c Q (t) = {c Q>Q (t),c hQ (t),--;c N _ hQ (t)} 



denote the spreading sequences for 0 < t < T s , which are applied to the in-phase and quadrature 
channels of the modulator. 

Consider the following set of orthonormal basis functions: 



1 1 k 



{ o, 



-cos(2tc/ c 0, kT<t<(k+l)T c 



otherwise 
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^sin((27i/ c O, ) kT c <t<(k+l)T c 



0, 



otherwise 



where T c is the chip duration; k = 0, 1, 2, ■ • •, JV-1 , and TV = TIT C , that is, N is the number of chips 
per bit. 

The DS/QPSK modulated signal can be written as follows (using the set of basis functions): 
S(t) = ± i^-J^ cos ^0c / (0±^-J^s 1 n(27t/ c 0c e (0 



N-l 



N-l 



•^2N 5X^(0 ±^ Sc a4 (,)+ (0 

*=0 £=0 



The channel output at the receiving end of the system has the following form 
x(t) = s(t)+j(t) 

where j{t) denotes the interference signal. We may express the interference signal using the 
2AMimensional basis functions as follows: 



N-l 



N-l 



k=0 



k=0 



where 



c (k+DT b 



J <o. k = It, ^cj t)dt 



k = 0,1, AM 
The average power of the interferer is given by 
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, AM N-l 

T 1 v< .2 1 v, .2 

J = T b Zjc, k + r b LJc Q 

b k=Q b k=0 l ' k 



Assuming that the power is equally distributed between the in-phase and quadrature components: 



2(AM) 



3 = h s 4, 



The mean of the interference signal is zero. The variance of the interference signal is therefore 

2(AM) 



a 



jam 



1 y 2 

k=0 



Demodulation 



c,(t) 



x{t) 



" e (0 



c Q (t) 



Bandpass 




BPSK 


filter 


> 


demodulator 



-*— o- 



Multiplexer 



I Bandpass 




BPSK 


filter 


► 


demodulator 



-> — o- 



Estimate of 
original 
binary 
sequence 



There are two stages of demodulation. First, the received signal x(t) is multiplied by the 
despreading sequences c,(r) and c Q {t), yielding 

U ' (t) = ± ^ cos ( 2n fc t ) ±c i(t)c Q (t)^sm(2nf c t) + c [ (t)j(t) 



V) = ± fi sin ( 2n fc t ) ± c Q (t)c I (t)^co S (2nf c t) + c Q (t)j(t) 
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The second terms in the right-hand side of uft) and u Q {t) are filtered by the bandpass filters, and 
the BPSK demodulators recover estimates of their respective binary sequences. Finally, the 
multiplexer reconstructs the original binary data stream. 

Processing gain 

The signal-to-noise ratio at the output of the receiver is 
(SNR) 0 = Instantan eous peak signal power 



2 

°jam 



2E 



JT/2 JT C 

The signal-to-noise ratio at the input of the coherent receiver is 

(SNR) = avera g e input-signal power 
1 average interferer power 

= E/T = E_ 
J ~ JT 

We may therefore write 
-(SNR) 



lOlog 

10 



.(SNR)J 
The QPSK processing gain= T/T 

2T 



b 



c 



That is, 

■^qpsk = 2[PG BK¥ ~\ 



Solving for the antenna aperture: 
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Problem 7.12 



The processing gain (PG) is 



Hence, expressed in decibels, 



PQ = FH bandwidth 
symbol rate 

We 

= 5 x 4 = 20 

PG = 101og 10 20 
= 26 db 



Pmh1em7.13 



The processing gain is 



PG = 4 x 4 
= 16 



Hence, in decibels, 



PG = 101og 10 16 
= 12 dB 
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Problem 7.13 



Matlab codes 

*/. Problem 7.13(a), CS: Haykin 

'/. Generating 63-chip PN sequences 

*/, polynomiall(x) = x"6 + x + 1 

'/. polynomial^ (x) = x~6 + x~5 + x*2 + x + 1 

'/, Mathini Sellathurai, 10.05.1999 

'/, polynomials 
poll=[l 0 0 0 0 1 1]; 
pol2=[l 10 0 111]; 

'/, chip size 
N=63; 

'/, generating the PN sequence 
pnseql = PNseq(poll); 
pnseq2 = PNseq(pol2) ; 

*/. mapping antipodal signals (0 — >-l, 1 — >1) 

u=2*pnseql-l ; 

v=2*pnseq2-l; 
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'/. autocorrelation of pnseql 
[corrf]=pn_corr(u, u, N) 

'/, prints 

plot(-61:62,corrf (2:125)); axis ([-62, 62,-10, 80]) 
xlabel(' Delay \tau') 

ylabel(' Autocorrelation function R_{c}(\tau) ' ) 
pause 

'/.autocorrelation of pnseq2 
[corrf]=pn_corr(v, v, N) 

'/, prints 

plot(-61:62,corrf (2:125)); axis ([-62, 62,-10, 80]) 
xlabelC Delay \tau') 

ylabelC Autocorrelation function R_{c}(\tau) ' ) 
pause 

'/. cross correlation of pnseql, pnseq2 
[c_corr]=pn_corr(u, v, N) 

'/. prints 

plot(-61:62,c_corr(2:125)); axis([-62, 62,-20, 20]) 
xlabel(' Delay \tau') 

ylabelC Cross-correlation function R_{j i}(\tau) ' ) 
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'/. Problem 7.13 (b) , CS: Haykin 

'/. Generating 63-chip PN sequences 

'/. polynomial l(x) = x~6 + x + 1 

*/. polynomial2(x) = x~6 + x"5 + x~2 + x + 1 

'/. Mathini Sellathurai, 10.05.1999 

'/, polynomials 
poli=[l 110 0 11]; 
pol2=[l 10 0 111]; 

'/. chip size 
N=63; 

'/. generating the PN sequence 
pnseql = PNseq(poll); 
pnseq2 = PNseq(pol2) ; 

*/. mapping antipodal signals (0 — >-l, 1 — >1) 

u=2*pnseql-i; 

v=2*pnseq2-l ; 

'/, autocorrelation of pnseql 
[corrf]=pn_corr(u, u, N) 

'/. prints 

plot (-61: 62, corrf (2:125)); axis ([-62, 62,-10, 80]) 
xlabelC Delay \tau') 

ylabelC Autocorrelation function R_{c}(\tau) ' ) 
pause 

'/.autocorrelation of pnseq2 
[corrf]=pn_corr(v, v, N) 

'/. prints 

plot(-61:62, corrf (2:125)); axis ([-62, 62,-10, 80]) 
xlabel(' Delay \tau') 

ylabel(' Autocorrelation function R_{c>(\tau) ' ) 
pause 

'/. cross correlation of pnseql, pnseq2 
[c_corr]=pn_corr(u, v, N) 

'/, prints 
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plot(-61:62,c_corr(2:125)); axis([-62, 62,-20, 20]) 
xlabelC Delay \tau') 

ylabel(' Cross-correlation function R_{j i}(\tau) ' ) 
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function x = PNseq(p) 

*/. Linear shift register for generating PN sequence of polynomial p 
*/, used for problems 7.13, 7.14 of CS: Haykin 
'/. Mathini Sellathurai, 10.05.1999 

N = length(p) - 1; */, order of the polynomial 

p = fliplr(p); 

X = [1 zeros(l, N-l)] ; 

n = 1; 

for i = 1 : n*(2~N - 1) 
x(i) = X(l); 

X = [X(2:N) p(N+l) * rem(sum(p(l:N) .* X(1:N)), 2)]; 
end 
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0 



function [corrf]=pn_corr(u, v, N) 

'/. funtion to compute the autocorreation/ cross-correlation 

'/, function of two PN sequences 

*/. used in problem 7.13, 7.14, CS: Haykin 

7. Mathini Sellathurai, 10 june 1999. 

max_cross_corr=0 ; 

for m=0:N 

shifted_u=[u(m+i:N) u(l:m)]; 
corr(m+l) = (sum(v.*shifted_u)) ; 
if (abs(corr)>max_cross_corr) 
max_cross_corr=abs (corr) ; 
end 
end 

corrl=f lipud(corr) ; 
corrf=[corrl(2:N) corr]; 
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Answer to Problem 7.13 




o 

Delay x 



Figure I : Autocorrelation function of [6, 5, 2,1], [6,1] 
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Figure 2- Cross-correlation function of [6, 5, 2,1], [6,1] 
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Figure §: Cross-correlation function of [6,5, 2,1], [6,5,4,1] 
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Problem 7.14 



Matlab codes 

'/. Problem 7.14 (a), CS : Haykin 

7i Generating 31-chip PN sequences 

'/. polynomiall(x) = x"5 + x~2 + 1 
'/. polynomial2(x) = x"5 + x~3 + 1 
*/. Mathini Sellathurai, 10.05.1999 

'/. polynomials 
poll=[l 0 0 10 1]; 
pol2=Cl 0 10 0 1]; 

'/, chip size 
N=31; 

'/, generating the PN sequence 
pnseql = PNseq(poll); 
pnseq2 = PNseq(pol2) ; 

'/. mapping antipodal signals (0 — >-l, 1 — >1) 
u=2*pnseql-l ; 
v=2*pnseq2-l ; 

'/, cross correlation of pnseql, pnseq2 
[c_corr]=pn_corr(u, v, N) 

'/, prints 

plot(-30:31,c_corr); axis([-30, 31,-15, 15]) 
xlabelC Delay \tau') 

ylabel(' Cross-correlation function R_{ji}(\tau) ' ) 
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'/. Problem 7.14 (b) , CS: Haykin 

*/. Generating 63-chip PN sequences 

7. polynomiall(x) = x~5 + x~3 + 1 

'/. polynomial2(x) = x~5 + x~4 + x~2 + x + 1 

'/. Mathini Sellathurai, 10.05.1999 

'/, polynomials 
poll=[l 0 10 0 1]; 
pol2=[l 10 111]; 

'/, chip size 
N=31; 

'/. generating the PN sequence 
pnseql = PNseq(poll); 
pnseq2 = PNseq(pol2) ; 

*/, mapping antipodal signals (0 — >-i, 1 — >1) 
u=2*pnseql-l ; 
v=2*pnseq2-l ; 

'/, cross correlation of pnseql, pnseq2 
[c_corr]=pn_corr(u, v, N) 

'/, prints 

plot(-30:31,c_corr) ; axis([-30, 31,-10, 10]) 
xlabelC Delay \tau') 

ylabelC Cross-correlation function R_{ji}(\tau 



432 



'/. Problem 7.14 (c), CS: Haykin 

'/, Generating 63-chip PN sequences 

'/. polynomiall(x) = x~5 + x"4 + x"3+l 

'/. polynomial2(x) = x*5 + x~4 + x~2 + x + 1 

*/. Mathini Sellathurai, 10.05.1999 

*/, polynomials 
poll=[l 1110 1]; 
pol2=[l 10 111]; 

'/, chip size 
N=31; 

'/, generating the PN sequence 
pnseql = PNseq(poll); 
pnseq2 = PNseq(pol2) ; 

'/. mapping antipodal signals (0 — >-l, 1 — >1) 
u=2*pnseql-l ; 
v=2*pnseq2-l ; 

'/, cross correlation of pnseql, pnseq2 
[c_corr]=pn_corr(u, v, N) 

'/. prints 

plot(-30:31,c_corr) ; axis([-30, 30,-10, 10]) 
xlabelC Delay \tau') 

ylabelC Cross-correlation function R_{ji}(\tau) ' ) 
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Answer to Problem 7.14 




Figure /: Cross-correlation function of [5, 3], [5, 2] 
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Delay t 



Figure Z - Cross-correlation function of [(5,5, 2,1], [6,1] 




Delay x 



Figure Cross-correlation function of [(5, 5, 2,1], [6, 5,4,1] 
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CHAPTF.R 8 



Problem 8.1 

(a) Free space loss = lOlog 



_ m , 4x7ixl50 

= 201 °§ J 5 a dB 



loV 3 x 10°/4 x 10 9 
= 88 dB 
(b) The power gain of each antenna is 

101og 10 G r = 101og 10 G ? = lOlog f4xnxA 



v \ 2 
io v A 



1A1 ,'4X71X71X0.6 

= lOlog 



io v (3/40) 2 
= 36.24 dB 

(c) Received Power= Transmitted power +G r - Free space loss 

= 1 +36.24- 88 
= -50.76 dBW 

Problem 8.2 

The antenna gain and free-space loss at 12 GHz can be calculated by simply adding 201og 10 (12/4) 
for the values calculated in Problem 8.1 for downlink frequency 4 GHz. Specifically, we have: 

(a) Free-space loss= 88 + 201og 10 (3) 

= 97.54 dB 

(b) Power gain of each antenna 

= 36.24 + 201og 10 (3) 
= 45.78 dB 

(c) Received power = -50.76 dBW 

The important points to note from the solutions to Problems 8.1 and 8.3 are: 

1. Increasing the operating frequency produces a corresponding increase in free-space loss, and an 
equal increase in the power gain of each antenna. 

2. The net result is that, in theory, the received power remains unchanged. 
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Problem 8.3 



The Friis free-space equation is given by 



(a) Using the relationship 



X 1 X 2 
A r = ~^G r > an< ^ A t = ~^G t ' we ma y wr ite 



P - P 

1 r 1 t 



X 2 



'4nA r 



■ x ■ 

And 



X 2 d 2 



(1) 



(b) P r = P, 



'4nA t 

IF 



— T 

4ndj 



4nd 2 



(2) 



In both Eqs. (1) and (2) the dependent variable is the received signal power, but the 
independent variables are different. 

(c) Equation (1) is the appropriate choice for calculating P r performance when the dimensions of 
both the transmitting and receiving antennas are already fixed. Equation (1) states that for 
fixed size antennas, the received power increases as the wavelength is decreased. 

Equation (2) is the appropriate choice when both A t and G r are fixed and the requirement is to 
determine the required value of the average transmitted power P t in order to realize a specified 

Pr 

Problem 8.4 



The free space loss is given by 
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■Mree space — I ^ J 

According to the above formulation for free space loss, free space loss is frequency dependent. 
Path loss, as characterized in this formulation, is a definition based on the use of an isotropic 
receiving antenna (G r =1). 

The power density, p(d), is a function of distance and is equal to 
EIRP 



9(d) = 



And 2 



The received power of an isotropic antenna is equal to 
P r = p(d) x 



4n 



eirp 

47tJ 2><47r 



EIRP 

47t<fj 2 



- EIRP/L free . space (1) 

Equation (1) states the power received by an isotropic antenna is equal to the effective transmitted 
power EIRP, reduced only by the path loss. However, when the receiving antenna is not isotropic, 
the received power is modified by the receiving antenna gain G n that is, Eq. (1) is multiplied by 
G r 

Problem 8.5 

In a satellite communication system, satellite power is limited by the permissible antenna size. 
Accordingly, a sensible design strategy is to have the path loss on the downlink smaller than the 
pass loss on the uplink. Recognizing the inverse dependence of path loss on the wavelength X, it 
follows that we should have 

^■uplink < ^downlink 

or, equivalently, 

/uplink > ./downlink 
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Problem 8.6 



Received power in dBW is defined by 
Pr = EIRP + G r - Free-space loss 
For these three components, we have 
(1) EIRP = lOlog JP t G t ) 



101og 10 P,+ 101og 10 (G,) 



= 101og 10 (0.1) + 101og 10 (G t ) 



Transmit antenna gain (in dB): 



101og 10 G, = lOlog 



4 x n x 0.7 x n/ 4 
10V (3/40) 2 



= 30.89 dB 



(2) Receive antenna gain: 



101og 10 G r = lOlog 



/ 2\ 

4 X7t x 0.55 xn x 5 



10 



(3/40)" 



= 49.84 dB 



(3) Free-space loss: 



L p = 201og 



4 x 7t x R 



= 201og 



4x7tx4x 10 



lol 3/40 



= 196.25 dB 



Hence, using Eqs. (1) to (5), we find that 
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P r = 101ogio(0.1) + 30.89+49.84- 196.52 

= - 206.52 + 8.073 
= -125 dBW 
Problem 8.7 

(a) RMS value of thermal noise = Je[v 2 ] = j4kTRAf volts, where k is Boltzmann's constant 

equal to 1.38 x 10" , T is the absolute temperature in degrees Kelvin, and R is the resistance 
in ohmns. Hence, 

RMS value = Ja x 1.38 x 10~ 23 x 290 x 75 x 1 x 10 6 

= V4xl.38 x290x75 xl0" 17 

= 1.096 x 10" 6 volts 

(b) The maximum available noise power delivered to a matched load is 

kTAf = 1.38 x 10" 23 x 290 x 10 6 
= 4.0 x 10" 15 watts 

Problem 8.8 

The wavegude loss is 1 dB; that is, 

^waveguide = 0-78 

The noise temperature at the input to the LNA due to the combined presence of antenna and 
waveguide is 

T e ~ ^waveguide X -^antenna + ~ ^waveguide) -^waveguide 

= 0.78 x50 + 290(1 -0.78) 
= 102.8A: 

The overall noise temperature of the system is 
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system ^ + JU + 2 00 200 

= 160.3/sT 

The system noise temperature referred to the antenna terminal is 
160.3/0.78 = 205.5/sT 
Problem 8.9 

In this problem, we are given the noise figures (F) and the available power gains (G) of the 
devices. By using the following relationship, we can estimate the equivalent noise temperature of 
each device: 



T e = T(F-l) 

where T is room temperature (290K) and T e is the equivalent noise temperature. 

(a) The equivalent noise temperatures of the given four components are 
Waveguide 

^waveguide = 290(2- 1) 

= 29QK 

Mixer 

^m.xer = 290(3- 1) 

= 580AT 

Low-noise RF amplifier 
T RF = 290(1.7-1) 

= 203 K 

IF amplifier 
T lF = 290(5- 1) 

= 1160JT 

(b) The effective noise temperature at the input to the LNA due to the antenna and waveguide is 
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T e ^waveguide X ^"antenna + ( ^ ^waveguide) X ^waveguide 

= 0.2x50 + 290(1-0.2) 
= 242tf 

The effective noise temperature of the system is 



T T 
T — T -i- T I mixer MF 

J system ~ 1 e + 1 RF + 7^ ^7=T 



'RF ^RF ^ ^mixer 



= 242 + 203 + ^0 + 1160 



10 10x5 
= 526.2AT 
Problem 8.10 



(a) For the uplink power budget, the ratio— is given by 



C 

N uplink 

where 



= ^-G^BOj + ^-k-L, 



§ s = Power density at saturation 

Gj = Gain of lm 2 

BO { = Power back-off 

— = Figure of Merit 

k = Boltzmann constant in dBK 

L r = Losses due to rain 



For the given satellite system, we have 

= - 81 - 44.5 - 0.0 + 1.9 + 228.6 + 0.0 



C 

N 



uplink 

= 105.0 dB-Hz 
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where we have used the following gain of lm antenna: 



mi i 4X7tXl 

Gj = lOlog 



10 X3xl0 8 /14xl0 9 ) 2 



= 44.5 dB 
Boltzmann constant k = -228.6 dB 



(b) Given the data rate in the uplink = 33.9 Mb/s and link margin of 6 dB, the required 



if) = {f) -(101og 10 M + 101og 10 /?) 

y W required vV uplink 



= 105 - 6 - 101og 10 (33.9 x 10 6 ) 

= 105-6-75.3 
= 23.7 dB 
Equivalently, we have 

E b 

TT = 234 

Given the use of 8-PSK, the symbol error rate is defined by 



P e = erfcl /-^ sin (71/8) 



= 234x3 = 702 



For 8-PSK 

E _ 3E b 
No N 0 

Hence, 



P e = erfc(V702x sin(7t/8)) 
= 0 
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This result further confirms the statement we made in Example 8.2 in that the satellite 
communication system is essentially downlink-limited. Recognizing that we have more powerful 
resources available at an earth station then at a satellite, it would seem reasonable that the BER at 
the satellite can be made practically zero by transmitting enough signal power along the uplink. 

Problem 8.11 

For the downlink, the relationship between 



where M is the margin and R is the bit rate in bits/second. 

Solving Eq. (1) for the the link margin in dB and evaluating it for the problem at hand, we get 
101og 10 M = 85-10-101ogio(10 6 ) 
= 5dB 

For the downlink budget, the equation for ( ] , expressed in decibels, is as follows: 



where k is Boltzmann's constant. 

For a satisfactory reception at any situation, we consider additional losses due to rain etc. up to the 
calculated link margin of 5 dB. Hence, we may write 




, expressed in decibels, is described by 




(1) 






'freespace 



101og 10 fc-101og 10 M(dB) 



(2) 



where 



E1RP = 57 dBW 



freespace = free-Space loss 
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= 92.4 + 201og 1() (12.5) + 201og 10 (40, 000) 

= 206 dB 
101og 10 £ = 228.6 dBK 

101og 10 M = 5 dB 

Using these values in Eq. (2) and solving for GJT, we get 



V 1 7dB 

= 10.4 dB 
WithT=310K, we thus find 
G r = 10.4+ 101og 10 (310) 

= 35.31 dB 
The receiving antenna gain in is given by 



For a dish antenna (circular) with diameter D, the area A equals 7tD 2 /4. Thus, 
101og 10 G r = 201og 10 £> + 201og 1( /+101og 10 (Ti) + 20.4(dB) 

where D is measured in meters and /is measured in GHz. Solving for the antenna diameter for the 
given system, we finally get 

Anin = 0-6 meters 

Problem 8.12 

(a) Similarities between satellite and wireless communications: 

• They are both bandwidth-limited. 

• They both rely on multiple-access techniques for their operation. 

• They both have uplink and downlink data transmissions. 

• The performance of both systems is influenced by intersymbol interference 
and external interference signals. 
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(b) Major differences between satellite and wireless communications: 



• Multipath fading and user mobility are characteristic features of wireless 
communications, which have no counterparts in satellite communications. 

• The carrier frequency for satellite communications is in the gigahertz range 
(Ku-band), whereas in satellite communications it is in the megahertz range. 

• Satellite communication systems provide broad area coverage, whereas wireless 
communications provide local coverage with provision for mobility 

in a cellular type of layout. 

Problem 8.13 

In a wireless communication system, transmit power is limited at the mobile unit, whereas no 
such limitation exists at the base station. A sensible design strategy is to make the path loss (i.e., 
free-space loss) on the downlink as small as possible, which, in turn, suggests that we make 

(Path loss) uplink < (Path loss) downlink 

Recognizing that path loss is inversely proportional to wavelength, it follows that 

^■uplink > ^ downlink 

or, equivalently, 

/uplink ^/downlink 
Problem 8.14 

The phase difference between the direct and reflected waves can be expressed as 



2nd 

4> = 



(i) 



X 

where X is the wave length. For large d, Eq. (1) may be approximated as 

, *K(h b h m ) 

9 ~ r— — radians 

Ad 

With perfect reflection (i.e., reflected coefficient of the ground is -1) and assuming small 0 (i.e., 
large d), the received power P r is defined by 
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°{ Id ) 



(2) 



where P Q = P t G b G t 




(3) 



Using Eq. (3) in (2): 




Id J \4nd) 




which shows that the received power is inversely proportional to the fourth power of distance d 
between the two antennas. 

Problem 8.15 

The complex (baseband) impulse response of a wireless channel may be described by 



where the amplitudes a x and a 2 are Rayleigh distributed, and the phase angles § l and ty 2 are 
uniformly distributed. This model assumes (2) the presence of two different clusters with each 
one consisting of a large number of scatterers, and (2) the absence of line-of-sight paths in the 
wireless environment. Define 

h(t) = h(t)e 1 
0 = § 2 -§ x 
We may then rewrite Eq. (1) in the form 

h(t) = a l 6(t-T) + a 2 e~ jQ 8(t-x) 

as stated in the problem. 

(a) (i) The transfer function of the model is 



h(t) = a,e 



§{t-x) + a 2 e &{t-x) 



(1) 
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H(f) = F[h{t)\ 



-727I/T, -7(2ji/t 2 + G) 



- a x e + a 2 e 



(ii) The power-delay profile of the model is 



P h = E[\h(tf] 



E[a x §(t - x,) + a 2 e jQ 8(t - x 2 )(a 1 8(? - x)) + a 2 e 7 °5(? - x 2 ) ] 
2 2 2 2 

EfajS (r-x,) + a 2 8 (?-x 2 ) + <3 1 a 2 cos08(r-x,)5(r-x 2 )] 

£[a 2 ]5 2 (r — ii ) + £[a 2 ]5 2 (f - t 2 ) (1) 



(b) The magnitude response of the model is 

Iu/Al -fl*f*\ -J2nf(x 2 + Q) 

\H(f)\ - a x e +a 2 e 

= ^a, + a 2 + 2a,a 2 cos(27i/'(x 2 - x^ + 6) 

which exhibits frequency selectivity due to two factors: (1) variations in the coefficients aj and a 2 . 
and (2) variations inthe delay difference x 2 -Xj. 

Problem 8.16 

The multipath influence on a communication system is usually described in terms of two effects - 
selective fading and intersymbol interference. In a Rake receiver, selective fading is mitigated by 
detecting the echo signals individually, using a correlation method, and adding them algebraically 
(with the same sign) rather than vectorially, and intersymbol interference is dealt with by 
reinserting different delays into the various detected echoes so that they fall into step again. 

Making each correlator perform at its assigned value of delay can be done by inserting the right 
amount of delay in either the reference (called the delayed-reference) or received signals (called 
the delayed- signal). Independent of the form of the reference signals employed, the output SNR 
from the integrating filters is substantially the same for both configurations, under the assumption 
that the length of the delay T d is significantly smaller than the symbol duration T. Each integrating 
filter responds to signals only within about +1/7 of the frequency /. Therefore, the noises adding 
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shorter than T, regardless of the form of reference signal. The only difference between the tap 
circuit contributions of the delayed-signal scheme and those of the delayed-reference scheme is 
that the latter are staggered in time by various fractions of T d , and since such staggering is small 
compared to the significant fluctuation period of the contributions, we conclude that the noise 
outputs of the two configurations are equivalent. 

However, there are three practical advantages of the delayed-signal scheme over the delayed- 
reference scheme. First, one delay line instead of two is required. Second, in the latter 

configuration, corresponding taps in the mark (symbol 0) and space (symbol 1) lines would have 
to be adjusted and be kept in phase coincidence. Third, coherent intersymbol interference 
(eliminated in the delayed-signal scheme) is still present in the latter scheme, (Price and Green 
1958), see the Bibliography. 

Problem 8.17 

(a) The output of the linear combiner is given by 



N 



N 



= ^a ; (z ; m(0 + n/0) 



N N 



Y J U j Zjm(t) + Y j a- j n j {t) 



Y ' ' V 

signal noise 



The output signal-to-noise ratio is therefore 



(SNR) 0 = 



Average signal power 
Average noise power 



r N 



2 



E Xa,.(z;ro(/)) 



r n 



2 



E X«^(0 
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r N N 

-j=u=i - 

■ N N 
■j=lk=l 



N N 



_ j=\k=\ 



N N 

SE a A E[nj(t)n k (t)] 
j=lk=\ 



Using the following expectations 



E[m (t)] = 1 for all t (i.e., unit message power) 



Eln jW n k (,)] = h fm k = j 

I 0 for k * j 



we find that Eq. (1) simplifies to 



(SNR) 0 = 



N N 

7=1 £=1 



N 



E2 2 



/ <V \2 



E2 2 

7=1 



(b) Equation (2) can be rewritten in the equivalent form 



450 



(SNR) 0 = 



N 



■7=1 



N 



E2 2 



N 



S"5 



1/2 



(3) 



2 , 2 



where m ; - = a ; - Oy and (SNR) y - = z ; -/<J y - • We now invoke the Schwarz inequality, which, in 
discrete form for the problem at hand, is stated as follows 



f N 



5>/SNR)j 

\j=i 



1/2 



^ 2 



f N \ 



f N 



1/2.2 



2((SNR)/ Z ) 

v ; =i A 7 =i 



(4) 



Hence, inserting this inequality into the right-hand side of Eq. (3), we may write 



N 

(SNR) 0 <£(SNR) y . 

7=1 

which proves the formula under subpart (i). 

To prove subpart (ii), we recall that the Schwarz inequality of Eq. (4) is satisfied with the 
equality sign if (except for a scaling factor) 

1/2 



(SNR)j = Uj 
or, equivalently, 



-J- = a a 
a , J 3 



That is, 
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Problem 8.18 



For the problem at hand, we have M = N = 2. Therefore, 
M-N+l = 1 

and so the weight subspace Wis one-dimensional. We thus have the following representation for 
the action of the antenna array: 



C 2 

(Interferer) 




Weight 



One-dimensional 
sub-space W 



Problem 8.19 



(a) The cost function is 



1 1 ,2 1 * 

J = ^e[n]\ = 2 e ^ e 



where the error signal is 

M 

e\n] - d[n]- ^w^nU^n] 
k=\ 

Let 

w k [n] = a k [n] + jb k [n\ 
Hence 



dJ 1 r ~.de [n] 1 * .de[n] 
5— = ~e[n] * + -=e [n]^-^ 
aa k 2 aa k 2 oa k 
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1 * 1 * 

= --e[n]x k [n] - -e [n]x k [n] 



1 * 
--Re{x k [n]e [n]} 



dJ 1 r ,de [n] 1 * Mn] 



= ~e[n]x* k [n] + ^e*[n]x k [n] 



= -lm{x k [n](e [«])} 
The adjustment applied to the kth weight is therefore 
Aw k [n] = Aa k [n] + jAb k [n] 

dJ . dJ 

= -^- ] »w k 

where jx is the step-size parameter. Substituting Eqs. (1) and (2) into (3), 
Aw k [n] = \xRe{x k [n]e [n] } + \ilm{x k [n]e*[n] } 

= \ix k [n]e [n] 

(b) The complex LMS algorithm is described by the following pair of relations 
wjn+l] = w k [n] + Aw k [n] 

= w k [n] + \ix k [n]e*[n], k=\,2,---,M 

M 

e(n) = d[n]-^w k x k [n] 
k=l 
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Problem 8.20 

(a) We are told that the speed of response of the weights in the LMS algorithm is proportional to 
the average signal power at the antenna array input. Conversely, we may say that the average 
signal power at the array input is proportional to the speed of response of the weights in the 
LMS algorithm. Moreover, the maximum speed of response of the LMS weights is 
proportional to /V/max> where R b is the bit rate and / max is the maximum fade rate in Hz. It 
follows therefore that the dynamic range of the average signal power at the antenna array 
input is proportional to RiJf m&x , as shown by 



where a is the proportionality constant, 
(b) For a = 0.2, R b = 32\ 10 3 b/s, and/ max = 70 Hz, the use of Eq. (1) yields 

P max = 0.2x32x 10 3 /70 

= 640/7 
= 91 watts 
which is somewhat limited in value. 
Problem 8.21 

(a) According to the Wiener filter, derived for the case of complex data, the optimum weight 
vector is defined by 



max 



watts 



(1) 



(1) 



where 



R x = correlation matrix of the input signal vector x[n] 



= E[x[n]x" [n]] 



(2) 



= cross-correlation vector between x[n] and desired response d[n] 



* 



= E[x[n]d [«]] 



(3) 



w c = optimum weight vector. 
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Note that the formulation of Eq. (1) is based on the premise that the array output is defined as 

the inner product w H x[n]. The Wiener filter for real data is a special case of Eq. (1), where the 
Hermitian transpose H in Eq. (2) is replaced by ordinary transposition and the complex 
conjugation in Eq. (3) is omitted. Assuming that the input \[n] and desired response d[n] are 
jointly ergodic, we may use the following estimates for R x and r xd : 



k=l 



(4) 



K 

^x[n]d[n] (5) 



where K is the total number of snapshots used to train the antenna array. Correspondingly, the 
estimate of the optimum weight vector w 0 is computed as 

w = C'f x , (6) 
where kJ is the inverse of R x . 

(b) The DMI algorithm for computing the estimate w may now proceed as follows: 

1. Collect K snapshots of data denoted by 

{x[k],d[k]}f =l 

where K is sufficiently large for w to approach w 0 and yet small enough to ensure 
stationarity of the data. 

2. Use Eqs. (4) and (5) to compute the correlation estimates R x and r xd . 

3. Invert the correlation matrix R v and then use Eq. (6) to compute the weight estimate w . 

For an antenna array consisting of M elements, the matrix R x is an M-by-M matrix and r xd is 
an M-by-1 vector. Therefore, the inversion of K x and its multiplication by f x ^ requires 
multiplications and additions on the order of M 3 . 
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Chapter 9 



Fundamental limits in Information Theory 



Problem 9.1 



Amount of information gained by the occurrence of an event of probability p is 



I = log 2 



T 



bits 



I varies with p as shown below: 



iQ - 




0 0.1 0.4 0.6 0.8 1.0 
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Problem 9.2 

Let the event denote the emission of symbol s^ by the source, Hence, 



I(s k ) = log 2 



bits 





s 0 


s l 


s 2 


s 3 


Pk 


0.4 


0.3 


0.2 


0.1 


I(s k ) bits 


1.322 


1.737 


2.322 


3.322 



Problem 9.3 

Entropy of the source is 





( ~\ 




( \ 




( \ 






H(S) = p 0 log 2 


1 


+ Pll<>g2 


1 




1 


+ P3 lo S2 


1 



= Ilog 2 (3) + Ilog 2 (6) + 1 log 2 (4) + 1 log 2 (4) 
3 6 4 4 

= 0.528 + 0.431 + 0.5 + 0.5 

= 1.959 bits 



Problem 9.4 

Let X denote the number showing on a single roll of a die. With a die having six faces, we note that 
Px is 1/6. Hence, the entropy of X is 



H(X) = p x log 2 



( \ 

v pI, 



= _ log 2 (6) = 0.431 bits 
6 
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Problem 9.5 



The entropy of the quantizer output is 



H = - Y, P(X k ) log 2 P(X k ) 
k=l 



where denotes a representation level of the quantizer. Since the quantizer input is Gaussian with 
zero mean, and a Gaussian density is symmetric about its mean, we find that 

P(X!) = P(X 4 ) 
P(X 2 ) = P(X 3 ) 

The representation level X 1 = 1.5 corresponds to a quantizer input +1 < Y < <». Hence, 



P(Xi) = f~ _L exp 



( \ 

-z! 



dy 



-I-Ierf| 



f \ 
4 



2 2 
= 0.1611 

The representation level X2 = 0.5 corresponds to the quantizer input 0 < Y < 1. Hence, 



P(x 2 ) = £ 



1 1 



J2k 



exp 



f \ 

-z! 



dy 



= 1 erfl 



f \ 
4 



2 

= 0.3389 

Accordingly, the entropy of the quantizer output is 



H = - 2 



0.1611 log 2 



= 1.91 bits 



' 1 ^ 
0.1611 
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0.3389 log 2 (0.3389) 



Problem 9.6 



(a) For a discrete memoryless source: 

P( 0i ) = P( Sil ) P( Sii ) ... P(s in ) 
Noting that M = K 11 , we may therefore write 



M-l M-l 

E P^) = £ P( Sii ) P(s i2 ) ... P(s in ) 
i=0 i=0 



K-l K-l K-l 

E E E P( 8il ) P(s i2 ) ... P(s in ) 

ij=0 i 2 =0 i n = 0 

K-l K-l K-l 

E pk> E P(s i2 > - E p(s in ) 

i 1= 0 i 2 =0 i n =0 



= 1 



(b) For k = l,2,...,n, we have 



M-l 

E P(o- t ) log 2 



i=0 



M-l 

E P(s h ) P(s i2 ) - P(s in ) log 2 
i=0 



( \ 
1 



For k=l, say, we may thus write 



M-l 


( \ 


K-l 


E P(°i)iog 2 


1 


- E 


i=0 


Pil 


i=0 




^ 1 ) 








K-l 

= E 






i=0 
= H(S) 



( \ K-l 



, Pi i , 
V 1 J 



E P(s i2 ) - E P(s in ) 



i=0 



i=0 



( \ 



Clearly, this result holds not only for k=l but also k»2,...,n. 
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(c) 



M-l 

H(S n )= £ P^lofe * 
i=o P(ot> 

M-l 

= y: p(°i)iog2 



« P( Sil ) P(s i2 ) - P(s in ) 



= l lp(Oi)l0g2 p^ + l lp(Oi)l0g2 p^ 

M-l 

+ ••• + E m) iog 2 ^ 

i=0 ns V 



Using the result of part (b), we thus get 



H(S n ) = H(S) ♦ H(S) + - + H(S) 
= nH(S) 



Problem 9.7 " 

(a) The entropy of the source is 



H(S) = 0.7 log 2 — + 0.15 log 2 1 + 0.15 log 2 1 



0.7 0.15 0.15 

= 0.258 + 0.4105 + 0.4105 
= 1.079 bits 

(b) The entropy of second-order extension of the source is 

H(S 2 ) = 2 x 1.079 = 2.158 bits 
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Problem 9.8 



The entropy of text is defined by the smallest number of bits that are required, on the average, to 
represent each letter. 

According to Lucky* English text may be represented by less than 3 bits per character, because 
of the redundancy built into the English language. However, the spoken equivalent of English text 
has much less redundancy; its entropy is correspondingly much higher than 3 bits. It follows 
therefore from the source coding theorem that the number of bits required to represent (store) text 
is much smaller than the number of bits required to represent (store) its spoken equivalent. 

Problem 9.9 

(a) With K equiprobable symbols, the probability of symbol &^ is 



Pk = P(s k ) = ^ 

The average code -word length is 



K-l 

r = £ Pk!k 
k=0 

1 K_1 
= ^E Ik 
K k=0 

The choice of a fixed code-word length l k =l 0 for all k yields the value L=l 0 . With K symbols in the 
code, any other choice for l k yields a value for L no less than 1 0 . 

(b) Entropy of the source is 
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R.W. Lucky, "Silicon Dreams", p.lll (St. Martin's Press, 1989). 



K-l 

h(S) = y: Pk iog 2 

k=0 
K-l 



f \ 
_L_ 

Pk 



= E l lo ^K = log 2 K 
k=0 K 



The coding efficiency is therefore 



H(S) = log2K 



For T|=l, we have 



lo = log2 K 



To satisfy this requirement, we choose 



K = 2 



where 1 0 is an integer. 



Problem 9.10 

A prefix code is defined as a code in which no code work is the prefix of any other code word. By 
inspection, we see therefore that codes I and IV are prefix codes, whereas codes II and III are not. 

To draw the decision tree for a prefix code, we simply begin from some starting node, and extend 
branches forward until each symbol of the code is represented. We thus have: 



0_ 



-O &. 



Coda. 
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Problem 9. 11 



We may construct two different Huffman codes by choosing to place a combined symbol as low or 
as high as possible when its probability is equal to that of another symbol. 

We begin with the Huffman code generated by placing a combined symbol as low as possible: 




The source code is therefore 



s 0 


0 


s l 


1 1 


s 2 


10 0 


s 3 


10 10 


s 4 


10 11 
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The average code-word length is therefore 

4 

E = E Pk!k 
k=0 

= 0.55(1) + 0.15(2) + 0.15(3) + 0.1(4) + 0.05(4) 
=1.9 

The variance of L is 

o 2 - E Pkd k - D 2 

k=0 

= 0.55(-0.9) 2 + 0.15(0.1) 2 + 0.15(1.1) 2 + 0.1(2.1) 2 + 0.05(2.1) 2 
= 1.29 

Next placing a combined symbol as high as possible, we obtain the second Huffman code: 



0 




O.os JJ 



Correspondingly, the Huffman code is 



s 0 0 



Si 100 



101 465 



s 3 110 
s 4 111 



The average code-word length is 



T = 0.55(1) + (0.15 + 0.15 + 0.1 + 0.05) (3) 
= 1.9 

The variance of L is 

o 2 = 0.55(-0.9) 2 + (0.15 + 0.15 + 0.1 + 0.05) (l.l) 2 
= 0.99 

The two Huffman codes described herein have the same average code-word length but different 
variances. 



Problem 9.12 
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The Huffman code is therefore 



s 0 


1 0 


s l 


1 1 


s 2 


00 1 


s 3 


0 10 


s 4 


Oil 


s 5 


0000 


s 6 


0001 



The average code-word length is 



k=0 

= 0.25(2)(2) + 0.125X3)(3) + 0.0625(4)(2) 
= 2.625 



The entropy of the source is 



H(S) = £ p k log 2 

k=0 

= 0.25X2) log 2 



J_ 
Pk 

( 



0.25 



0.125(3) log 2 



/ 1 ^ 



0.125 



) 



0.0625(2) log 2 



\ 



0.0625 



= 2.625 

The efficiency of the code is therefore 



= H(S) = 2.625 = x 
r 2.625 

We could have shown that the efficiency of the code is 100% by inspection since 
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where l k = loggCl/pj^). 

Problem 9.13 
(a) 

s 0 0.7 

s, Qi5 

5 2 0.15 

The Huffman code is therefore 

s 0 0 
b 1 10 
s 2 11 

The average code-word length is 

L = 0.7(1) + 0.15(2) + 0.12(2) 
= 1.3 



(b) For the extended source we have 



Symbol 


s o s o 


S 0 S 1 


S 0 S 2 


s l s 0 




8 i s i 


s l s 2 


S 2 S 1 


S 2 S 2 


Probability 


0.49 


0.105 


0.105 


0.105 


0.105 


0.0225 


0.0225 


0.0225 


0.0225 
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Applying the Huffman algorithm to the extended source, we obtain the following source code: 



s o s o 


-1 
1 


S 0 S 1 


00 1 


S 0 S 2 


0 10 


s l s 0 


Oil 


S 2 S 0 


0000 


s l s l 


000100 


s 1 s 2 


000101 


s 2 s 1 


000110 


S 2 S 2 


000 111 



The corresponding value of the average code-word length is 



= 0.49(1) + 0.105(3)(3) + 0.105(4) + 0.0225(4)(4) 
= 2.395 bits/extended symbol 

-L = 1.1975 bits/symbol 



(c) The original source has entropy 



H(S) = 0.7 log 2 
= 1.18 



,0.7, 



0.15(2) log 2 



' 1 ^ 
0.15 



According to Eq. (J0-.28), 



H(S) < ^ < H(S) + 1 
n n 

This is a condition which the extended code satisfies. 



Problem 9.14 



Symbol 



Huffman Code 



Code-word length 



A 
B 
C 
D 
E 
F 
G 



1 

Oil 
0 10 
00 1 
0 0 11 
0000 1 
00000 



1 

3 
3 
3 
4 
5 
5 



Problem 9.15 

00 
t i 

01 
10 



t 

2. 



4 

2 



0 



0 



\ 



Computer code Probability Huffman Code 



0 0 1 0 

2 

11 1 10 

4 

0 1 1 no 

8 

10 1 HI 

8 



The number of bits used for the instructions based on the computer code, in a probabilistic sense, 
is equal to 470 



'1.1 1.1^ 



+ + _^ + 

,2 4 8 8, 



= 2 bits 

On the other hand, the number of bits used for instructions based on the Huffman code, is equal to 



1x1+2x1 + 3x1+3x1 = 1 
2 4 8 8 4 

The percentage reduction in the number of bits used for instruction, realized by adopting the 
Huffman code, is therefore 



100 x i^i = 12.5% 



Problem 9.16 
Initial step 

Subsequences stored: 0 

Data to be parsed: 11101001100010110100 
Step 1 

Subsequences stored: 0, 1, 11 

Data to be parsed: 101001100010110100.. 
Step 2 

Subsequences stored: 0, 1, 11, 10 

Data to be parsed: 1001100010110100 .... 

Step 3 

Subsequences stored: 0, 1, 11, 10, 100 

Data to be parsed: 1100010110100... 

Step 4 

Subsequences stored: 0, 1, 11, 10, 100, 110 
Data to be parsed: 0010110100... 
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Step 5 

Subhsequences stored: 
Data to be parsed: 



0, 1, 11, 10, 100, 110, 00 
10110100 .... 



Step 6 

Subsequences stored: 0, 1, 11, 10, 100, 110, 00, 101 
Data to be parsed: 1 0 1 0 0 ... 



Step 7 

Subsequences stored: 0, 1, 11, 10, 100, 110, 00, 101, 1010 
Data to be parsed: 0 .... 

Now that we have gone as far as we can go with data parsing for the given sequence, we write 



Numerical 123 4 5 6 7 8 9 

positions 

Subsequences 0, 1, 11, 10, 100, 110, 00, 101, 1010 

Numerical 22, 21, 41, 31, 11, 42, 81 

representations 

Binary encoded 0101, 0100, 0100, 0110, 0010, 1001, 10000 

blocks 
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Problem 9.17 




p(y 0 ) = (1 - p)p(xo) + p ptei) 

= (1 - p) (1) + p(.I) 

= _1 
2 



p(yi) = p p(xo) + (i - p) p(xi) 
= p(^) + (i - P ) ( i ) 

= 2 

2 
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Prnh1em9.18 



P(x 0 ) 
P(xi) 



4 
4 



P(y 0 > = (l - p) (1) + p(I) 

4 4 

= i + 2 
4 2 

pCyj) = p(i) + (1 - p) d) 
4 4 

-1-2 

4 2 



Problem 9.19 

From Eq.(^*52.)we may express the mutual information as 



1 1 ( ■ • ^ 

kx,y) = £ J) p< x j»yk) iog 2 

j=0 k=0 



p(xj»yk) 



^p(xj) p(y k ) ^ 

The joint probability p(xj,y k ) has the following four possible values: 

j = k = 0: p(xo ,y k ) = p 0 (l-p) = (1 - p x ) (1 - p) 

j = 0, k = 1: p(x 0 ,y 1 ) = P() p = (1 - Pl )p 

j = 1, k = 0: p(x 1>yo ) = p x p 

j = k = 1: p(x 1 ,y 1 ) = p(l - )a) 

where p 0 = p(x 0 ) and Pl = jKxj) 

The mutual information is therefore 
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I(X;Y) = (1- Pl ) (1-p) log 2 



(1- Pl ) (1-p) 



(1- Pl ) ((1- Pl ) (1- P ) + Pl p) 



+ (1- P i) P log 2 



(1-Pl) P 



(1- Pl ) ((1- Pl ) P + Pl (l- P )) 



PiP log 2 



PiP 



Pi(U-Pi) (1-p) + PiP) 



Pl (l-p) log 2 



Pid-P) 



Pi((l-Pi)p + Pid-P) 



Rearranging terms and expanding algorithms: 



I(X;Y) = P log 2 P + (1- P ) log 2 (l- P ) 

- [Pid-P) + d-Pi)p] log 2 [pi(l-p) + (1-Pi)p] 

- [PiP + (1-Pl) (1-P)] log 2 [PiP + (1-Pi) (1-p)] 

Treating the transition probability p of the channel as a running parameter, we may carry out the 
following numerical evaluations: 

p=0: 

I(X;Y) = - p x log 2 Px - (1 - p x ) log 2 (1 - p x ) 
Pi = 0.5, I(X;Y) = 1.0 

p=01 : 

I(X;Y) = - 0.469 - (0.1 + 0.8 p x ) log 2 (Oil + 0.8 p x ) 

- (0.9 - 0.8 p x ) log 2 (0.9 - 0.9 Pi) 
p 1 = 0.5, I(X;Y) = 0.531 

p=0.2 : 

I(X;Y) = - 0.7219 - (0.2 + 0.6 p^log^j (0.2 + 0.6 p x ) 

- (0.8 - 0.6 P!> log 2 (0.8 - 0.6 pj) 
Pl = 0.5, I(X;Y) = 0.278 
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p=0.3 : 



I(X;Y) = - 0.88129 - (0.3 + 0.4 p^ log 2 (0.3 + 0.4 p x ) 

- (0.7 - 0.4 Pl ) log 2 (0.7 - 0.4 Pl ) 
p x = 0.5, I(X;Y) = 0.1187 

p=0.5 : 

KX;Y) = 0 

Thus, treating the a priori probability pj as a variable and the transition probability p as a running 
parameter, we get the following plots: 



1. 0 r P=0 




Problem 9.20 

From the plots of I(X;Y) versus p 1 for p as a running parameter, that were presented in the solution 
to Problem 10rl9 we observe that I(X;Y) attains its maximum value at p 1 =C5for any p. Hence, 
evaluating the mutual information I(X;Y) at p 1 =0.5, we get the channel capacity: 



C = 1 + p log 2 p + (1 - p) log 2 (1 - p) 
= 1 - H(p) 



where H(p) is the entropy function of p. 
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Problem 9.21 

(a) Let 

z = pj(l - p) + (1 - p x ) p = (1 - p 0 ) (1 - p) + p 0 P 

Hence, 

PlP + (1 - Pi) - 1 - Pi (1 - p) - (1 - Pi) P 

= 1 - z 

Correspondingly, we may rewrite the expression for the mutual information I(X;Y) as 

I(X;Y) = H(z) - H(p) 

where 

H(z) = - z log 2 z - (1 - z) log 2 (1 - z) 
H([) = - p log 2 p - (1 - p) log 2 (1 - p) 

(b) Differentiating I(X;Y) with respect to p 0 (or pj) and setting the result equal to zero, we find that 
I(X;Y) attains its maximum value at p 0 = p 1 = 1/2. 

(c) Setting p 0 = Pj - 1/2 in the expression for the variable z, we get: 

z = 1 - z = 1/2 

Correspondingly, we have 

H(z) = 1 

We thus get the following value for the channel capacity: 

C = I(X;Y) l, o = pi = 1/2 
= 1 - H(p) 

where H(p) is the entropy function of the channel transition probability p. 
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Problem 9.22 




From this diagram, we obtain (by inspection) 

P(y 0 Ixq) = (1 - p) 2 + p 2 = 1 - 2p(l - p) 
P(y 0 Ixi) = p(l - p) + (1 - p)p = 2p(l - p) 

Hence, the cascade of two binary symmetric channels with a transition probability p is equivalent 
to a single binary symmetric channel with a transition probability equal to 2p(l - p), as shown below: 



I - 1p0~f>) 




Correspondingly, the channel capacity of the cascade is 



C = 1 - H(2p(l - p)) 

= 1 - 2p(l - p) log 2 [2p(l - p)] - (1 - 2p - 2p 2 ) log 2 (l - 2p - 2p 2 ) 
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Problem 9.23 



1 - a 




1 -a 



The mutual information for the erasure channel is 



l 2 



(1) 



The joint probabilities for the channel are 



P( x o>yo) = 0 -°OPo 



P(x v y 0 ) = 0 



P(x 0 ,y 2 ) = p 0 a 



p(x 0 ,y x ) = 0 



P(x v yi) = (l-a)p l p(x v y 2 ) = Pl a 



where p Q + pi = 1. Substituting these values in (1), we get 



I(X;Y) = (1-a) 



. Poloe {7J + (1 - po)log lT 



Since the transition probability p - (1 - a) is fixed, the mutual information I(X;Y) is maximized 
by choosing the a priori probability p 0 to maximize H(p 0 )- This maximization occurs at p 0 = 1/2,. 
for which H(p 0 ) = 1. Hence, the channel capacity C of the erasure channel is 1 - a. 
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Problem 9.24 

(a) When each symbol is repeated three times, we have 
Messages Unused signals Channel outputs 



001 


000 


010 


001 


Oil 


010 


100 


100 


101 


101 


110 


110 




111 



We note the following: 

1. The probability that no errors occur in the transmission of three 0s or three Is is (1 - p) 3 . 

2. The probability of just one error occurring is 3p(l - p) 2 . 

3. The probability of two errors occu.no »gi s 3p 2 (l - p). 

4. The probability of receiving all three bits in error is p 3 . 

With the decision-making based on a majority vote, it is clear that contributions 3 and 4 lead to the 
probability of error 

P 3 = 3p 2 (l-p) + p 3 

(b) When each symbol is transmitted five times, we have 

Messages Unused signals 
00000 

00001 
00010 
00011 

11110 

11111 



Channel outputs 

00000 

00001 

00010 

00011 



11110 
11111 
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The probability of zero, one, two, three, four, or five bit errors in transmission is as follows, 
respectively: 



(1-p) 5 
5p (1-p) 4 
10p 2 (l-p) 3 
lOp^l-p) 2 
5p 4 Cl-^ 

„5 



The last three contributions constitute the probability of error 

P e = P 5 + 5p\l-p) + 10p 3 (l-p) 2 

(a) For the general case of n=2m + 1, we note that the decision-making process (based on a majority 
vote) makes an error when m+1 bits or more out of the n bits of a message are received in error. The 



probability of i message bits being received in error is 
Hence, the probability of error is (in general) 



n 



pXI-p) 1 



,n-i 



n 



Pe= E 

i=m + l 



Pid-P) 



n-i 



The results derived in parts (a) and (b) for m=l and m=2 are special cases of this general formula. 

Problem 9.25 

The differential entropy of a random variable is independent of its mean. To evaluate the differential 
entropy of a Gaussian vector X, consisting of the components X 1 ,X 2 ,...,X n , we may simplify our task 
by setting the mean of X equal to zero. Under this condition, we may express the joint probability 
density function of the Gaussian vector X as 
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f x (x) = 



(v^") n 



exp 



°1°2- -On 



2ot 2<Jo 



2a' 



The logarithm of fyix) is 



log 2 f x (x) = - log^TC)" 72 0x02...^) - 



2of 20o 



2af 



log 2 e 



Hence, the differential entropy of X is 



h(X) = -//-../ fx(x) log 2 (fx(x)) dx 

= log 2 ((2^) n/2 a 1 o 2 ...a n ) j f ... J tyx)dx 



We next note that 



2 2 2 

2 a£ 



2of 2a 2 



fx(x)dx 



J J... Jfx(x)dx = 1 
//... Jxff^dx-af 



i = l,2,...,n 



Hence, we may simplify (1) as 
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h(X) = log 2 [(2ji) n/2 o 1 o 2 ...o n ] + 21log 2 e , 



n/2 



+ £ log 2 e 



2n(al4 ... o 2 n ) yn 



* L log 2 e 



- | log 2 



27ie(o 2 o 2 , ... o 2 ^ 11 



When the individual variances are equal: 



2 <2 2 2 

a, = * 0 = ...a„ = a 



(ojoj ... o y n = o 2 



Correspondingly, the differential entropy of X is 



h(X) = ^ log 2 (2jteo 2 ) 



Problem 9.7.6 



(a) The differential entropy of a random variable X is 



h(X) = - J~ fx(x) log 2 fx(x)dx 



The constraint on the value x of the random variable X is 



|x l< M 



Using the method'Lagrange multipliers, we find that h(X), subject to this constraint, is maximized 
when 
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Xm["^ x) log 2 fx(x) + fx(x)]dx 



is stationary. Differentiating -fxCxfloggf^x) + Af^x) with respect to f^x), and then setting the result 
equal to zero, we get 

-log 2 e + X = log 2 f x (x) 

This shows that f^x) is independent of x. Hence, for the differential entropy h(X) under the 
constraints I x I < M and f~ fx(x)dj<»| to be maximum, the random variable X 



(b) The maximum differential entropy of the uniformly distributed random variable X is 



must be uniformly distributed: 



1/2M, 
. 0, 



-M < x < M 



fx(x) = 



otherwise 



h(X) = _L_ log 2 (2M)dx 
J-M 2M 




= log 2 (2M) 
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Problem 9.27 



(a) 



I(X;Y) = J_~ J_~ fx, Y (x,y)log 2 



f x ( x |y) 



f X (x) 



dxdy 



I(Y;X) = £ f ~ fx, Y (x,y)log 2 



f Y (y k) 



f Y (y) 



dxdy 



From Bayes' rule, we have 



Hence, I(X;Y) = I(Y;X). 



(b) We note that 



f x (x ly) _ fy(y k) 
^x«~ = fy(y) 



f x,Y( x »y) = f x( x ly)fv(y) 



Therefore, we may rewrite (1) as 



I(X;Y) = f~ f~ f XY (x,y)log 5 

— oo — oo * 



f x,Y< x »y) 



f x (x)fY(y) 



dxdy 



From the continuous version of the fundamental inequality, we have 



|_~ fx,Y(x,y)log 2 



f ^ 

f x,Y( x »y) 



v f x (x)f y (y) j 



dxdy > 0 



which is satisfied with equality if and only if 
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Hence, 



f x,Y< x >y) = fx( x ) f Y<y) 



I(X;Y) > o 



(c) By definition, we have 



h(X |Y) = f~ f~ f XY (x,y)log 2 



1 



dxdy 



By definition, we also have 



h(X) = J~ f x (x)log 2 



f X (x) 



dx 



Since 



J_~ f Y (y bc)dy = 1, 
we may rewrite the expression for h(X) as 



h(X) = f~ f x log 2 _i_ dx f~ f^ybOdx 
J -°° f x (x) 



= J_~ fy(y f X (x)log 2 



dxdy 



But 



f Y (y lx) f x (x) = f X) y(x,y) 
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Hence, 



h(X) = J~ J_~ fx, Y (x,y)log 2 



dx 



Next, we subtract (3) from (4), obtaining 



h(X) - h(X |Y) = f~ f~ f XY (x,y) 

J—oo J— CO * 



log2 ■ log2 W 



dxdy 



= f~ f°° f X ,Y< x >y) lo g2 

J—oo J—oo 9 

= KX;Y) 



fx(x) 



dxdy 



(d) Using the symmetric property derived in part (a), we may also write 



I(Y;X) = h(Y) - h(Y |X) 



Problem 9 7.8 



By definition, we have 



h(Y |X) = J*~ fj^yHoga 



1 

fy(y b0 



dxdy 



Since 



f x,v< x >y) = f Y<y lx)fx( x ) 



we may also write 
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h(Y |X) = J_~ f x (x)dx J_~ fy(y |x)log 2 



1 

WE) 



dy 



(1) 



The variable Y is related to X as 



Y = X + N 

Hence, the conditional probability density function f Y dy I x) is identical to that of N except 
for a translation of x, the given value of X. Let f N (n) denote the probability density function 
of N. Then 



fyCy bO = f N (y-x) 



Correspondingly, we may write 



= X" f N (n)log 2 



%(y-x) 

dn 



dy 



f N (n) 



(2) 



= h(N) 



where, in the second line, we have used n = y-x and dn = dy for a given x. Substituting fe^) 
in (1): 



h(Y |X) = h(N) £ f x (x)dx 
= h(N) 
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Problem 9.29 



(a) Channel bandwidth B = 3.4 kite: 

Received signal-to-noise ratio SNR = 10 3 = 30 dB 

Hence, the channel capacity is 

C = B log 2 (l + SNR) 

= 3.4 x 10 3 log 2 (l + 10 3 ) 
= 33.9 x 10 3 bits/second 

(b) 4800 = 3.4 x 10 3 log 2 (1 + SNR) 
Solving for the unknown SNR, we get 

SNR = 1.66 = 2.2 dB 

Problem 9.30 

With 1 0 distinct brightness levels with equal probability, the information in each level is 
log 2 10 bits. With each picture frame containing 3 x 10 5 elements, the information content 
of each picture frame is 3 x 10 5 log 2 10 bits. Thus, a rate of information transmission of 30 
frames per second corresponds to 

30 x 3 x 10 5 log^lO = 9 x 10 6 log 2 10 bits/second 
That is, the channel capacity is 



C = 9 x 10 6 log 2 10 bits/second 
the information capacity theorem: 489 



C = B log 2 (1 + SNR) 
With a signal-to-noise ratio SNR = 10 3 = 30 dB, the channel bandwidth is therefore 



B = 



C 



log 2 (l + SNR) 



9 x 10 6 log 2 10 



log^ 1001 
= 3 x 10 3 Hz 



Problem 9.31 

The information in each element is log 2 10 bits. 

The information in each picture is [3 log 2 (10)] x 10 5 bits. 

The transmitted information rate is [9 log 2 (10)] x 10 6 bits/second. 

The channel must have this capacity. From the information capacity theorem, 



c = B log 2 (l + SNR). 



Therefore, 



[9 log 2 (10)] x 10 6 bits/second = Blog 2 (l + 1000). 



Solving for B, we get 



B 



= 9 x 10 6 Hz • 



log 2 (10) ' 
log 2 (1001). 



= 3 x 10 6 Hz 
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Problem 9.32 

Figure 1 shows the 64-QAM constellation. Under the condition that the transmitted signal energy 
per symbol is maintained the same as that in Fig. 1, we get the tightly packed constellation of Fig. 
2. We now find that the Euclidean distance between adjacent signal points in the tightly packed 
constellation of Fig. 2 is larger than that of the 64-QAM constellation in Fig. 1. From Eq. (5.95) 
of the textbook, we recall that an increase in the minimum Euclidean distance results in a 
corresponding reduction in the average probability of symbol error. It follows therefore that, with 
the signal energy per symbol being maintained the same in the two constellations of Figs. 1 and 2, 
a digital communication systems using the tightly packed constellation of Fig. 2 produces a 
smaller probability of error than the corresponding 64-QAM system represented by Fig. 1. 



10 



-5 



0 - 



5 




-10 



-10 



-5 



0 



5 



10 
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Problem 9.33 



In the textbook, the capacity of the NEXT-dominated channel is derived as 



1 + 



NEXT 



(/)| 



2 \ 



\H(ft 



df 



where F a is the set of positive and negative frequencies for which S x (f) > 0, where S x (f) is the 
power spectral density of the transmitted signal. 

For the NEXT-dominated channel described in the question, the capacity is 



1 + 



3/2 



exp(-ajf) 



df 




where P, k, I and f 0 are all constants pertaining to the transmission medium. This formula for 
capacity can only be evaluated numerically for prescribed values of these constants. 



493 



Problem Q 34 

For k=l, Eq. (9./3S) reduces to 

10 log 2 (SNR) = 6 log 2 N + C 
Expressing Eq. (3.33) in decibels, we have 



10 log 2 (SNR) = 6R + 10 log 10 ' 



The number of bits per sample R, is denned by 

R = log 2 N 

We thus see that Eqs. (1) and (2) are equivalent, with 



Ci = 10 log 10 



< 3P A 



2 

^ m max j 
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Problem 9.35 



The rate distortion function and channel capacity theorem may be summed up 
diagrammatically as follows: 




According to the rate distortion theory, the data compression limit set by minimizing the 
mutual information I(X;Y) lies at the extreme left of this representation; here, the symbol 
Y represents the data compressed form of the source symbol X. On the other hand, 
according to the channel capacity theorem the data transmission limit is defined by 
maximizing the mutual information I(X;Y) between the channel input X and channel output 
Y. This latter limit lies on the extreme right of the representation shown above. 
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1 Problem 9. 3<© 

Matlab codes 



*/. Computer Problem in Chapter 9 

'/, Figure: The minimum achievable BER as a function of 

'/. Eb/NO for several different code rates using binary signaling. 

'/, This program calculates the Minimum required Eb/NO 

*/. for BPSK signalling at unit power over AWGN channel 

'/. given a rate and an allowed BER. 

'/, Code is based on Brandon's C code. 

'/. Ref : Brendan J. Frey, Graphical models for machine 

'/. learning and digital communications, The MIT Press. 

'/, Mathini Sellathurai 

EbNo=double( [7.85168, 7.42122, 6.99319, 6.56785, 6.14714, 5.7329, 5.32711, ... 
4.92926, 4.54106, 4.16568, 3.80312, 3.45317, 3.11902, 2.7981, 2.49337, 2.20617, ... 
1.93251, 1.67587, 1.43313, 1.20671, 0.994633, 0.794801, 0.608808, 0.434862, ... 
0.273476, 0.123322, -0.0148204, -0.144486, -0.266247, -0.374365, -0.474747, -0.5708, 
-0.659038, -0.736526, -0.812523, -0.878333, -0.944802, -0.996262, -1.04468, ... 
-1.10054, -1.14925, -1.18536, -1.22298, -1.24746, -1.27394, -1.31061, -1.34588, ... 
-1.37178, -1.37904, -1.40388, -1.42553, -1.45221, -1.43447, -1.44392, -1.46129, ... 
-1.45001, -1.50485, -1.50654, -1.50192, -1.45507, -1.60577, -1.52716, -1.54448, ... 
-1.51713, -1.54378, -1.5684]); 



rate= double ( 
9.914587e-01, 
9.334680e-01, 
7.881055e-01, 
6.211661e-01, 
4.232136e-01, 
2.661871e-01, 
1.733225e-01, 
1.002778e-01, 
5.698847e-02, 
3. 185243e-02 
1.941896e-02, 
N=66; 



[9.989372e-01, 
9.868898e-01, 
9.155144e-01, 
7.565174e-01, 
5.866480e-01, 
3.938277e-01, 
2.451079e-01, 
1.588591e-01, 
9. 150450e-02, 
5. 188306e-02, 

, 2.883246e-02 
1.764122e-02, 



9.980567e-01, 
9.804353e-01, 
8.946454e-01, 
7.238745e-01, 
5.525132e-01, 
3.653328e-01, 
2.251691e-01, 
1.453627e-01, 
8.347174e-02, 
4.675437e-02, 
, 2.606097e-02 
1.586221e-02, 



9.966180e-01, 
9.722413e-01, 
8.715918e-01, 
6.900430e-01, 
5. 188620e-01, 
3.382965e-01, 
2.068837e-01, 
1.326278e-01, 
7.598009e-02, 
4.239723e-02, 
, 2.332790e-02, 
1.444108e-02, 



9.945634e-01, .. 
9.619767e-01, 9. 
8.459731e-01, 8. 
6.556226e-01, .. 
4.860017e-01, 4. 
3. 129488e-01, 2. 
1.894274e-01, .. 
1.210507e-01, 1. 
6.886473e-02, 6. 
3.851637e-02, 3. 

2. 185325e-02, . 
1.314112e-02]); 



490156e-01, 
178003e-01, 

539652e-01, 
889799e-01, 

101504e-01, 
266875e-02, 
476062e-02, . 



b=double([le-5]); */. Allowed BER 
'/. Rate R (bits per channel usage) 

r=double([l/32, 1/16,0.1,0.2,0.3,0.4,0.5, 0.6, 0.7, 0.8,0.85,0.95]); 
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le=zeros(l, length (r)) ; '/, initialize buffer for Eb/NO 
for p=l : length(r) 
c = double(r(p)*(1.0+b*log(b)+(1.0-b)*log(1.0-b)/log(2.0))) ; 
i=N-l; 

'/, Minimum Eb/NO calculations 

wkilo ( (a>-0) & (c>-riito(i)) ) 

i=i-l; 

end 

i=i+l; 



if ( (i>0) I (i<N ) ) 

e =double( EbNo(i)+(EbNo(i-l)-EbNo(i))*(c-rate(i))/(rate(i-l)-rate(i)) 
le(p)=10*logl0((10-(e/10))*c/r(p)); 
display(le) 

else 

display ( 'values out of range') 

end 

end 

plot(10*logl0(r) ,le, '-') 
xlabel('Rate (dB)') 
ylabeK 'Minimum E_b/N_0 (dB)') 
axis([10*logl0(l/32), 0, -2 4]) 
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'/, Computer Experiment in Chapter 9 
'/. Program to create the figure for the minimum 
'/, Eb/NO needed for error-free communication 
'/. with a rate R code, over an AWGN channel 
'/, using binary signaling 

V. Tkie program calcula/toe tko Minimum required Eb/NO 

'/. for BPSK signalling at unit power over AWGN channel 

'/, given a rate and an allowed BER. 

'/, Code is based on Brandon's C code. 

'/, Ref : Brendan J. Frey, Graphical models for machine 

'/, learning and digital communications, The MIT Press. 

7. Mathini Sellathurai 



EbNo= double ([7. 85 168, 7.42122, 6.99319, 6.56785, 6.14714, 5.7329, 5.32711 

4.92926, 4.54106, 4.16568, 3.80312, 3.45317, 3.11902, 2.7981, 2.49337, 2.20617, ... 

1.93251, 1.67587, 1.43313, 1.20671, 0.994633, 0.794801, 0.608808, 0.434862 

0.273476, 0.123322, -0.0148204, -0.144486, -0.266247, -0.374365, -0.474747, -0.5708, 
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-0 . 659038 . -0.736526. -0 .812523 . .-0 . 878333 . -fl. 944802 



iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiima 
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-1.51713, -1.54378, -1.5684]); 



iTiMiniL 

5065.4 , 



Pi 

MIL* 

.... imflmnmilr™ 1 

1" 50192, 



1p 

iiiiiiiiii imnit 



-0.996262. -1.04468 
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1.45B07, 60S77, -1 .527=16, -1.B4448, 



rate=double( [9 . 989372e-01 , 9 . 980567e-01 , 9 . 966180e-01 , 9 . 945634e-01 , ... 

9.914587e-01, 9 . 868898e-01 , 9 . 804353e-01 , 9 . 722413e-01 , 9 . 619767e-01 , 9 . 490156e-01 , 

9.334680e-01, 9 . 155144e-01 , 8 . 946454e-01 , 8 . 715918e-01 , 8 . 459731e-01 , 8 . 178003e-01 , 

7.881055e-01, 7 . 565174e-01 , 7 . 238745e-01 , 6 . 900430e-01 , 6 . 556226e-01 , ... 

6.211661e-01, 5 . 866480e-01 , 5 . 525132e-01 , 5 . 188620e-01 , 4 . 860017e-01 , 4 . 539652e-01 , 

4.232136e-01, 3 . 938277e-01 , 3 . 653328e-01 , 3 . 382965e-01 , 3 . 129488e-01 , 2 . 889799e-01 , 

2.661871e-01, 2.451079e-01, 2 . 251691e-01 , 2 . 068837e-01 , 1 . 894274e-01 , ... 

1.733225e-01, 1 . 588591e-01 , 1 .453627e-01 , 1 . 326278e-01 , 1 . 210507e-01 , 1 . 101504e-01 , 

1.002778e-01, 9 . 150450e-02 , 8 . 347174e-02 , 7 . 598009e-02 , 6 . 886473e-02 , 6 . 266875e-02 , 

5.698847e-02, 5 . 188306e-02 , 4 . 675437e-02 , 4.239723e-02, 3 . 851637e-02 , 3 . 476062e-02 , 

3. 185243e-02, 2 . 883246e-02 , 2 . 606097e-02 , 2 . 332790e-02 , 2.185325e-02 

1.941896e-02, 1 . 764122e-02 , 1 . 586221e-02, 1 .444108e-02, 1 . 314112e-02] ) ; 
N=66; 



b=double(0.5:-le-5:le-5); '/. Allowed BER 

rrr=double( [0.99, 1/2, 1/3, 1/4, 1/5, 1/8]); '/. Rate R(bits/channel usage) 
le=zeros (1, length (b)) ; 

for rr=l : length(rrr) 
r=rrr(rr) ; 

for p=l : length(b) 

c = double(r*(1.0+b(p)*log(b(p))+(1.0-b(p))*log(1.0-b(p))/log(2.0)) ) ; 
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i=N-l; 



while ( (i>=0) & (c>rate(i) ) ) 
i=i-l; 

end 

if ( (i>0) I (i<N ) ) 

e = double(EbNo(i)+(EbNo(i-l)-EbNo(i))*(c-rate(i))/(rate(i-l)-rate(i)) 
le(p)=10*loglO((10-(e/10))*c/r) ; 

else 

display ( 'values out of range') 

end 

end 

plot(le,10*logl0(b), '-') 

end 

xlabel('E_b/N_0 (dB)') 
y label ( 'Minimum BER') 
axis([-2 1 -50 -10]) 
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Answer to Problem 9.36 
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E b /N 0 < dB > 

Figure 2: The minimum achievable BER as a function of Eb/NO for several 
different code rates using binary signaling 
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Chapter 10 



Error-Control Coding 



Problem 10.1 



The matrix of transition probabilities of a discrete memoryless channel with 2 inputs and 
Q outputs may be written as 



P = 



p(o b) pd b) p(2 b) - p(Q-i b) 
P (o Id P (i Id p (2 Id - p cq-i Id. 



For a symmetric channel, 

p(j b) = p(Q-l - j ID, j = 0,1 Q-l 

Moreover, each row of the matrix P contains the same set of numbers, and each column of 
the matrix P contains the same set of numbers. For example, for Q=4, we may u**"L+<£_ 



P = 



a a b b 
b b a a 



The sum of the elements of each row of matrix P must add up to one. Hence, for this 
example, 



2a + 2b = 1 



The probability of receiving symbol j is 



p(j) = P(j b) p(0) + p(j ll)p(l) 



For equally likely input symbols: 
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p(0) = p(l) = ± 



Hence, 



p(j) = \ [p(j b) + P (Q-i - j k»] 



For the example of Q=4, we have 



P(j) = 



± (a + b) 
2 



j = 0,1,2,3 



In general, we may write 



J = 0,1 Q-l 



Problem 10.2 

For a binary PSK channel, the probability density function of the correlator output in the 
receiver is 



f x (x k» = 1 exp 
f x (x |l) = __L_ exp 



Let 



y pertains to a Gaussian variable of mean ± 
express the channel transition probability as 



2E> 



and unit variance. We may therefore 



p(y b) = — L_ exp 
J2k 



p(y |l) = exp 



1 

2 









2E b 


y + 


No J 


I < 





f 






1 






2E b 




y 




No J 


2 







where 



<y < 



0 


1 


1 


3 


-a. 


O 







4 
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p(0 b) = — f ~ exp 



1 

2 



f 






2E b 


y + 


No J 


I < 



dy 



= _ erfc 
2 



f 




a 


N„ J 



p(i b) = 



l ro 

J-a 6XP 



1 
2 



f 






2E b 


y + 


No J 


{ < 



dy 







r 












1 


erfc 






- erfc 


a 




Eb 


2 






No J 






\ 


N„ J 



p(2 b) = J- f a exp 
2 k •*> 



J. 
2 



\2 



2Eb 
N N o 



dy 



l 
2 



erfc 



^ ^ f 

Eb 
N 0 



erfc 



J 



a 



^2 > 



Nr 



p(3 b) = — f~ exp 



M2 



= — erfc 
2 





V 






_L + 






No J 



2Ej 
N N o 



dy 



We also note that 



p(3 1 0) = p(0 1 1) 
p(2|0) = p(l|l) 
p(l|0) = p(2|l) 
p(0 1 0) = p(3 1 1) 
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Hence, the channel is symmetric. 



Problem 10.3 



From the solution to Problem Id 7. , we readily note the following: 



p(y |0) = — — exp 
J2k 



p(y ll) = exp 
s[2k 





r 




1 




2E 


2 


y + 

I < 


No J 



< y < 





f 






1 










y 






2 




> 


N„ J 



< y < 



where E is the code symbol energy. 



Problem 10.4 

Message Sequence Single-p&vitY -check code 



0 00 


0 0 0 0 


0 0 1 


0 0 11 


0 10 


0 10 1 


Oil 


0 110 


10 0 


10 0 1 


10 1 


10 10 


110 


1100 


111 


1111 
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Problem 10.5 



For the (4,1) repetition code, the parity check matrix is 



1 0 0 j 1 
H = 0 1 0 j 1 
0 0 1 i 1 



For a (7,4) Hamming code, we have 



H = 



10 0 
0 10 
0 0 1 



10 11 
1110 
0 111 



For the Hamming code, the parity check matrix H is more structured than that for the 
repetition code. Indeed, the matrix H for the Hamming code includes that for the repetition 
code as a submatrix. 



Problem 10.6 

The generator matrix for the (7.4) Hamming code is 



G = 



1 


1 


0 


l 


0 


0 


0 


0 


1 


l 


0 


1 


0 


0 


1 


1 


l 


0 


0 


1 


0 


1 


0 


l 


0 


0 


0 


1 



The parity-check matrix is 
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H = 



Hence, 



10 0; 1011 
0 1 0 i 1110 
0 0 1; 0 111 



HG T = 















1 


0 


1 


1 


















1 


1 


1 


0 




1 


0 


0 


1 


0 


1 1 




0 


1 


1 


1 




0 


1 


0 


1 


1 


1 0 




1 


0 


0 


0 




p 


0 


1 


0 


1 


1 1_ 




0 


1 


0 


0 














0 


0 


1 


0 














.0 


0 


0 


1 




0 


0 


0 


0 
















0 


0 


0 


0 




mod- 


-2 










0 


0 


0 


0 















Problem 10.7 



(a) Viewing the matrix 



H = 



10 0 
0 10 
0 0 1 



10 11 
1110 
0 111 



as a generator matrix, we may define the code vector £ in terms of the message vector m 
as 



c = m H 



The message word length is 
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n - k = 7-4 = 3 



Hence, we may construct the following table 



Message word Code word Hamming weight 

000 0000000 0 

001 0010111 4 

010 0101110 4 

011 0111001 4 

100 1001011 4 

101 1011100 4 

110 1100101 4 

111 1110010 5 



(b) The minimum value of the Hamming weight defines the Hamming distance of the dual 
code as 

dmin = 4 



Problem 1 0 8 

(a) For a (5,1) repetition code: 



H = 



[1 


1 


1 


1 


1 : 


1 


0 


0 


0 


: 1 


0 


1 


0 


0 


': 1 


0 


0 


1 


0 


1 1 


0 


0 


0 


1 


I 1 


1 


0 


0 


0 




0 


1 


0 


0 




0 


0 


1 


0 




0 


0 


0 


1 




1 


1 


1 


1 
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The syndrome is 



s = e H T 

where e is the error pattern. For a single error, we thus have 



Error pattern Syndrome 

00001 1111 

00010 0001 

00100 0010 

01000 0100 

10000 1000 

(b) For two errors in the received word, we have 



Error pattern 


Syndrome 


0 0 0 1 1 


1110 


0 0 10 1 


110 1 


0 100 1 


10 11 


1 0 0 0 1 


0 111 


0 0 110 


0 0 11 


0 10 10 


0 10 1 


10 0 10 


10 0 1 


OH DO 


0 110 


10 10 0 


10 10 


11000 


110 0 



We note that the syndromes for all single-error and double-error patterns are distinct. This 
is intuitively satisfying since a (5,1) repetition code is capable of correcting up to two errors 
in the received vector 



y 



= e + c 



Problem 10.9 



g(X) = 1 + X + X 3 
c(X) = m(X)g(X) 

Hence, we may construct the following table: 



Message 


m(X) 


c(X) 


Code word 


word 








r\ n n n 


0 


0 


A A f\ f\ A A f\ 

0 0 0 0 0 0 0 


A A A "1 

U 0 0 1 


v3 


"v3 v4 v6 

X° + X + X" 


0 0 0 110 1 


A A 1 A 

U U 1 u 


v 2 
X 


v2 v3 v5 
X + X + X 


A A "1 "f A 1 A 

0 0 110 10 


0 10 0 


X 


X + X 2 + X 4 


0 110 10 0 


10 0 0 


l 


1 + X + X 3 


110 10 0 0 


0 0 11 


X 2 + X 3 


X 2 + X 4 + X 5 + X 6 


0 0 10 111 


0 110 


x + x 2 


X + X 3 + X 4 + X 5 


0 10 1110 


110 0 


1 + x 


1+X 2 + X 3 + X 4 


10 1110 0 


0 10 1 


x + x 3 


X + X 2 + X 3 + X 6 


0 1110 0 1 


10 10 


1+X 2 


1 + X + X 2 + X 5 


1110 0 10 


10 0 1 


1 +x 3 


1 + X + X 4 + X 6 


110 0 10 1 


0 111 


X + X 2 + X 3 


x + x 5 + X 6 


0 10 0 0 11 


1110 


1 + X + X 2 


l + X 6 *** 


10 0 0 110 


10 11 


1 + X 2 + X 3 


1+X+X 2 +X 3 +X 4 +X 5 +X 6 


1111111 


110 1 


1 + X + X 3 


1 + X* + X 6 


10 10 0 0 1 


1111 


1 + X + X 2 + X 3 


1+ X 3 + X 5 + X 6 


10 0 10 11 



Comparing the code word to the message word, we see that the cyclkcode generated by 
multiplying g(X) and c(X) is not a systematic code. 
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Problem 10.10 

Consider the generator polynomial 

g(X) = 1 + X 2 + X 3 
The encoder corresponding to this g(X) is as follows: 




The generator matrix G associated with this encoder is 



G + 



1 


0 


1 


1 


0 


0 


0 


0 


1 


0 


1 


0 


1 


0 


0 


0 


1 


0 


1 


1 


0 


0 


0 


0 


1 


0 


1 


1 



To reduce this matrix to a systematic form, we add row 1 to 2, add rows 1 and 2 to row 3, 
and add rows 2 and 3 to row 4: 
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G = 



1 0 

1 1 

1 1 

0 1 



1 
1 
0 

1 




For the syndrome calculator, we have 



Sec Ci'n/W 



4 



-4 



ftir. 



Given that 



G - p ' I 4 ] 

H = [l 3 . P T ] 
we find that the parity-check matrix is 



H = 



10 0J1110 
0 10 5 0 111 
0 0 1 i 1 1 0 1 



In Example 3 of the text, the message sequence (1001) was applied to the encoder and the 
output code word was 0111001. For the above encoder, the parity bits are 110 and the code 
word is then 1101001. In particular, we have 
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Shift Input Register contents 

000 

1 1 10 1 

2 0 111 

3 0 110 

4 1 110 



If we were to make an error in the middle bit and receive 1100001, then circulating it 
through the syndrome calculator, we have 



Shift 


Input 


Register contents 
000 


1 


1 


10 0 


2 


0 


0 10 


3 


0 


0 0 1 


4 


0 


10 1 


5 


0 


111 


6 


1 


0 10 


7 


1 


10 1 



From the parity check matrix we see that the syndrome calculator output 101 corresponds 
to the error pattern 0001000. The corrected code word is therefore 1101001. 



Problem 10 11 



The error polynomial is 



e(X) = r(X) + c(X) 

We are given 
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c(X) 
r(X) 

The error polynomial is therefore 



= X + X 2 + X 3 + X 6 
= X + X 3 + X 6 



e(X) = X 2 

Consider next the syndrome polynomial s(X). The syndrome calculator for the generator 
polynomial 

g(X) = 1 + X + X 3 

is shown in Fig. l&.ll; this calculator is reproduced here for convenience of presentation: 



0\o\oo\ 



Flip- 
flop 



Mod-2 
adder 



Circulating the received bits through the syndrome calculator, we may construct the 
following table: 

Initial state 0 0 0 

10 0 
0 10 
0 0 1 
0 10 
0 0 1 
0 10 



Here, the syndrome polynomial is 



s(X) = X 2 

which, for the problem at hand, is the same as the error polynomial. This result 
demonstrates the property of the syndrome polynomial, stating that it is the same as the 
error polynomial when the transmission errors are confined to the parity-check bits. In 
Problem 11.11 the third parity-check bit is received in error. 

Problem 10.12 

The encoder structure is 









- — ' « 


r 





The syndrome calculator is 




r4 
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Problem 10.13 



(a) A maximal-length code is the dual of the corresponding Hamming code. The generator 

polynomial of a (15,11) Hamming code is given as l+X + X 4 . We may therefore define the 
feedback connections of the corresponding (15,4) maximal-length code by choosing the 
primitive polynomial 

h{X) = l+X + X 4 

The feedback connections are therefore [4,1], which agrees with entry 3 of Table 7.1. 
Specifically, the encoder of the (15,4) maximal-length encoder is as follows: 



-4 

/VloJ-2. 



r- 1 1 p -ffop 

(b) The generator polynomial of the (15,4) maximal-length code is 



8(X) = 



l+X 
h(X) 



15 



l+X 



15 



l+X + X 



Performing this division modulo-2, we obtain 

g(X) = i+x + x 2 + x 3 + x 5 + x 7 + x\x 



ll 



(This computation is left as a[ exercise for the reader.) Assuming that the initial state of the 
encoder is 0001, we find that the output sequence is (111101011001000). Here we recognize 

that the length of the coded sequence is 2 4 - 1 = 15. The output sequence repeats itself 
periodically every 15 bits. 
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Prnhlem 10.14 

(a) n = 2*" -1 = 31 symbols 

Hence, the number of bits per symbol in the code is 
m = 5 bits 

(b) Block length = 31 x 5 = 155 bits 

(c) Minimum distance of the code is 

dmin =2t+l 
= n-k + 1 
= 31-15 + 1 
= 17 symbols 

(d) Number of correctable symbols is 

t = i(n - k) 
2 

= 8 symbols 
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Problem 10.15 



The encoder is realized by inspection: 

g (1) = (1,0,1) 
g (2) = (1,1,0) 
g (3) = (1,1,1) 

For the Hamming code, the parity check matrix H is more structured than that for the 
repetition code. Indeed, the matrix H for the Hamming code includes that for the repetition 
code as a submatrix. 




Message 1 0 
Output 11 10 
\ — 



11 11 
11 01 01 01 

Original message 
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The code is in fact systematic. 



Problem 10.17 



The generator polynomials are 

g (1) (X) = 1 + X + X 2 + X 3 
g (2) (X) = 1 + X + X 3 

The message polynomial is 

m(X) = 1 + X 2 + X 3 + X 4 + ... 

Hence, 

c (1) (X) = g (1) (X) m(X) 

= 1 + X + X 3 + X 4 + X 5 + 



c (2) (X) = g< 2 >(X) m(X) 

= 1 + X + X 2 + X 3 + X 6 + X' 

Hence, 
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{c (1) } = 1,1,0,1,1,1,... 
{c (2) } = 1,1,1,1,0,0,... 

The encoder output is therefore 11, 11, 01, 11, 10, 10. 



Problem 10.18 

The encoder of Fig. IM 3(b) has three generator sequences for each of the two input paths; 
they are as follows (from top to bottom) 



g x (1) = (1,1) gl (2) = (1,0), gl (3) = (1,1) 
9 2 (1) = (0,1), g 2 (2) = (1,1), g 2 (3) = (0,0) 



Hence, 

gl (1) (X) = i + x, gl (2) (X) = l, gl (3) (X) = i + x 

g 2 (1) (X) = X, g 2 (2) (X) = 1 + X, g 2 (3) (X) = 0 

The incoming message sequence 10111... enters the encoder two bits at a time; hence 
= 1 1 .... 
m (2) = 01 .... 
The message polynomials are therefore 
m^X) = 1 + X + ... 
m 2 (X) = X + ... 



Hence, the output polynomials are 
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c^X) = gl (1) (X) mi (X) + g 2 (1) (X) m 2 (X) 

= (1 + X)(l + X + ...) + X(X + ...) 

= 1 + .... 

c (2) (X) = gl (2) (X) mi (X) + g 2 (2) (X) m 2 (X) 

= (1) (1 + X + ...) + (1 + X) (X + ...) 

= i + x + ... + x + x 2 + ... 
= i + x 2 + ... 

c (3) (X) = gl (3) (X) mi (X) + g 2 (3) (X) m 2 (X) 

= (1 + X) (1 + X + ...) + (0) (X + ...) 

= l + x 2 + ... 

The output sequences are correspondingly as follows: 

c^ = 1, 0, ... 

c (2) = 1, 0, ... 

c (3) = 1, 0, ... 
The encoder output is therefore (1,1,1), (0,0,0), ... 

Problem 10.19 

1. If the input sequence is 00, the encoder output is 00, 00, 00, 00. 

2. If the input sequence is 11, the message polynomial is 

m(X) = 1 + X 
The two generator polynomials are 
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g (1) (X) = 1 + X + X 2 
g (2) (X) = 1 + X 2 

Hence, 

c (1) (X) = (1 + X) (1 + X + X 2 ) 

= 1 + X 3 
c (2) (X) = (1 + X) (1 + X 2 ) 

= 1 + X + X 2 + X 3 

The encoder output is 11, 01, 01, 11 

3. If the input sequence is 01, the message polynomial is 

m(X) = X 

Hence, 

c (1) (X) = X(l + X + X 2 ) 

= X + X 2 + X 3 
c (2) (X) = X(l + X 2 ) 

= x + x 3 

The encoder output is 00, 11, 10, 11 

4. Finally, if the input sequence is 10, the message polynomial is 

m(X) = 1 
Correspondingly, 
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c (1) (X) = g (1) (X) 

= 1 + X + X 2 
c®(X) = g( 2 )(X) 

= 1 + x 2 

Hence, the encoder output is 11, 10, 11, 00. 

The encoder outputs calculated above are in perfect accord with the entries of the code tree % 

OO OQ oo 

• — * a 

. 1 t . i Q . M 
1 • • a 

f O 0 N , OO 

m ■ • ■ • 
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Problem 10.20 



CO 
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The output sequence is 11, 10, 11, 01, 01 



Problem 10.21 




526 



Problem 10.22 



The encoder of Fig. PK>-/7has eight states: 



State Register contents 

a 0 0 0 

b 10 0 

c 0 10 

d 110 

e 0 0 1 

f 10 1 

g Oil 

h 111 



The state diagram of the encoder is as follows: 




In this diagram, a solid line corresponds to an input of 0 and a das bed line corresponds to 
an input of 1. 
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Problem 10.23 



(a) The encoder of Fig. /o-/3i> has four states: 



State Register contents 
a 0,0 
b 1,0 
c 0,1 
d 1,1 




(b) Starting from the all-zero state a, the incoming sequence 10111... produces the path a 

b d ... Equivalently, we have the decoded (output) sequence (111), (000) which is exactly 

the same result calculated in Problem 1(1.18. r 1s 



Problem 10.24 



An MSK system has two distinct phase states 



State Phase.radians 
a 0 
b 7i 

The transmission of a 1 increases the phase by ji/2, whereas the transmission of a 0 
decreases the phase by tc/2. Correspondingly, the transmission of dibit 10 or 01 leaves the 
state of MSK unchanged, whereas the transmission of dibit 00 or 11 movesthe system from 
one state to the other. For the output, we have 



Input dibit Output frequencies 
11 f x f x 
0 1 f 2 f, 

10 f x f 2 

oo f 2 f 2 



We may thus construct the following state diagram for MSK: 




The trellis diagram for MSK is as follows: 
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Since d^ = 5 and the number of errors in the received sequence is 2, it should be possible 
to decode the correct sequence. This is readily demonstrated by applying the Viterbi 
algorithm. 

Problem 10.25 




N/otcxV l OAS 

(n) p<xrR me-tAic 

message bit 0 

h\esso_j^ bit I 

j v. X.quajl toe See -that 

se^c* .'s 00 00 00 00 OO00 ^oxbj -AA-^ 

■the "tuJO ^o>S jke /-^.ca; vec^ .Se^^A^ , 
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Problem 10.26 



(a) Coding gain for binary symmetric channel is 



G a = 10 log 2 



'10 x 1/2^ 



= 10 log 10 2.5 
= 4 dB 

(b) Coding gain for additive white Gaussian noise channel is 



G a = 10 log 10 ^10 x 1 

= 10 log 10 5 
= 7 dB 



Problem 10.27 

The trellis of Fig. corresponds to binary data transmitted through a dispersive 

channel, viewed as a finite-state (i.e., two-state) machine. There are two states representing 
the two possible values of the previous channel bit. Each possible path through the trellis 
diagram of Fig. corresponds to a particular data sequence transmitted through the 

channel. 

To proceed with the application of the Viterbi algorithm to the problem at hand, we first 
note that there are two paths of length 1 through the trellis; their squared Euclidean 
distances are as follows: 

d l x = (1.0 - l.l) 2 = 0.01 

d l,2 = U- 0 " (-- 9))2 = 361 531 



Each of these two paths is extended in two ways to form four paths of length 2; their 
squared Euclidean distances from the received sequence are as follows: 

(a) 

d 21 = 0.01 + (0.0 - l.l) 2 = 1.22 
d 2 2 2 = 3.61 + (0.0 - 0.9) 2 = 4.42 

(b) 

d 23 = 0.01 + (0.0 - (-0.9)) 2 = 0.82 
d 2 2 4 = 3.61 + (0.0 - (-1.1)) 2 = 4.82 

Of these four possible paths, the first and third ones (i.e., those corresponding to squared 
Euclidean distances d 2 2 x and d 2 2 3 ) are selected as the "survivors", which are found to be 
in agreement. Accordingly, a decision is made that the demodulated symbol aQ=l. 



Next, each of the two surviving paths of length 2 is extended in two ways to form four new 
paths of length 3. The squared Euclidean distances of these four paths from the received 
sequence are as follows: 



(a) 



(b) 



da 2 ! = 1.22 + (0.2 - l.l) 2 = 2.03 
d 32 = 0.82 + (0.2 - 0.9) 2 = 1.31 



d 3 2 3 = 1.22 + (0.2 - (-0.9)) 2 = 2.43 
d 34 = 0.82 + (0.2 - (-1.1)) 2 = 2.51 
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This time, the second and third paths (i.e., those corresponding to the squared Euclidean 
distances d 2 ^^. and d 2 3 3 ) are selected as the "survivors". However, no decision can be 
made on the demodulated symbol aj as the two paths do not agree. 

To proceed further, the two surviving paths are extended to form two paths of length 4. The 
squared Euclidean distances of these surviving paths are as follows: 

(a) 

d 41 = 1.31 + (-1.1 - l.l) 2 = 6.15 
d 42 = 2.43 + (-1.1 - 0.9) 2 = 6.43 

(b) 

d 2 3 = 1.31 + (-1.1 - (-0.9)) 2 = 1.35 
d 44 = 2.43 + (-1.1 - (-1.1)) 2 = 2.43 

The first and third paths are therefore selected as the "survivors", which are now found to 
agree in their first three branches. Accordingly, it is decided that the demodulated symbols 
are a 0 = +1, &i = -1, and a 2 = +1. It is of interest to note that although we could not form 
a decision on aj after the third iteration of the Viterbi algorithm, we are able to do so after 
the fourth iteration. 

Figure 1 shows, for the problem at hand, how the trellis diagram is pruned as the 
application of the Viterbi algorithm progresses through the trellis of Fig. PI 1.5 
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(This problem is taken from R.E. Blahut, "Digital Transmission of Information", Addison- 
Wesley, 1990, pp. 144-149.The interested reader may consult this book for a more detailed 
treatment of the subject.) 
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Problem 10.29 



(a) Without coding, the required E b /N 0 is 12.5 dB. Given a coding gain of 5.1 dB, the required 
E)JNq is reduced to 



= 12.5-5.1 



req 



= 7.4 dB 



For the downlink, the equation for CINq is 



-p) = EIRP + % - L hee sm( . e + k 

iv (K downlink 1 



(b) By definition, the formula for receive antenna gain is 



47tA, 

x 



where A r is the receive antenna aperture and X is the wavelength. Let (A r ) coding denote the 
receive antenna aperture that results from the use of coding. Hence 



10 log 



id ; 2 



lOlog, 



4n(A r ) 



coding 



r 



5.1 dB 



or, equivalently, 



lOlog 



A, 



(^■r) coding 



5.1 dB 



Hence, 



^"^^codine 



= antilog 0.51 = 3.24 



coding 

The antenna aperture is therefore reduced by a factor of 3.24 through the use of coding. 
Expressing this result in terms of the antenna dish diameter, d,wc may write 
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nd 2 /4 



( 



d 



2 



= 3.24 



^coding) /4 



d. 



coding- 



which yields 



Diameter of antenna without coding _ d 



= 73.24 = 1.8 



Diameter of antenna with coding Coding 



That is, the antenna diameter is reduced by a factor of 1.8 through the use of coding. 
Problem 10.30 

Nonlinearity of the encoder in Fig. P10.30 is determined by adding (modulo-2) in a bit-by-bit 
manner a pair of sets of values of the five input bits {I\^ tv /2,«-i' h,n-2* h,iv h,n-\} anc * tne 
associated pair of sets of values of the three output bits Y 0 n , Y ln and Y 2 ir . If the result of adding 
these two sets of values of input bits, when it is treated as a new set of values of output-bits, does 
not always give a set of values of input bits identical to the result of adding the two sets of values 
of the aforementioned output bits, then the convolutional encoder is said to be nonlinear, For 
example, consider two sets of values for the sequence {/j „, I\ n .\, l\,n-2^ h,n> h,n-l^ ^at are S' ven 
by {0,0,1,1,1} and {0,1,0,0,0}. The associated sets of values of the three output bits F 0 „, Y x „, 
Y 2 j V are {0,1,1} and {1,0,0}, respectively. If the 5-bit sets are passed through the Exclusive OR 
(i.e., mod-2 adder) bit-by-bit, the result is {0,1,1,1,1 }. If the resulting set {0,1,1,1,1 } is input into 
the encoder, then the associated output bits are {1,1,0}. However, when the sets of output bits 
{0,1,1} and {1,0,0} are passed through the Exclusive OR, bit-by-bit, the result is {1,1,1}. Since 
the two results {1,1,0} and { 1,1,1 } are different, it follows that the convolutional encoder of Fig. 
P10.30 is nonlinear. 

Problem 10.31 

Let the code rate of turbo code be R. We can write 




1 

R 
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q l +q 2 ~P 



P 

Hence 

R = p/(q l + q 2 -p) 



Problem 10.32 

Figure 1 is a reproduction of the 8-state RSC encoder of Figure 10.26 used as encoder 1 and 
encoder 2 in the turbo encoder of Fig. 10.25 of the textbook. For an input sequence consisting of 
symbol 1 followed by an infinite number of symbols 0, the outputs of the RSC encoders will 
contain an infinite number of ones as shown in Table 1. 



Fig. 1 
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The output is 1011101001110100111... 
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Therefore, an all zero sequence with a single bit error (1) will cause an infinite number of channel 
errors. 

[Note: The all zero input sequence produces an all zero output sequence.] 
Problem 10.33 



(a) 4-state encoder 
x 




D 






D 





• ► 



-> x (systematic bits) 




_^ Parity check bits 



8-state encoder 
x 




^ x (systematic bits) 



^ Parity check bits 



16-state encoder 
x 



>■ (systematic bits) 




^ Parity check bits 
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(b) 4-state encoder 



1 +D + D 



21 



g(D) = 1 




By definition, we have 



\M{D)) 



( B(D) \ 




where 5(D) denotes the transform of the parity sequence {£,■} and M(D) denotes the transform 
of the message sequence {m,}. Hence, 

(l+D 2 )B(D) = (l+D + D 2 )M(D) 
The parity-check equation is given by 
(m- + m M + m._ 2 ) + (b i + b^) = 0 
where the addition is modulo-2. 

Similarly for the 8-state encoder, we find that the parity-check equation is 

m i + Wi-2 + m i-3 + b i + b i-\ + b i-2 + b i-3 = 0 

For the 16-state encoder, the parity-check equation is 

m i + m i-A + b i + b i-\ + b i-2 + b i-3 + b i-4 = 0 
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Problem 10.34 
(a) Encoder 
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GCj, oc 2 , • • •, a M are M interleavers 

ENCj, ENC 2 , ■ • - , ENC M are M recursive systematic convolutional (RSC) encoders 
z 0 is the message sequence 

z v z 2 , ■■■,z M are the resulting M parity sequences 
(b) Decoder 
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aj 1 , a 2 \ a m ! are de-interleavers. 
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The generalized encoder and decoder presented here are described in Valenti (1998); see the 
Bibliography. 



Problem 10.35 



The decoding scheme used for turbo codes relies on the assumption that the bit probabilities 
remain independent from one iteration to the next. To maintain as much independence as possible 
from one iteration to the next, only extrinsic information is fed from one stage to the next, since 
the input and the output of the same stage will be highly correlated. However, this correlation 
decreases as |^ - 1 2 \ increases, where t h t 2 are any two time instants. The interleaving is utilized to 
spread correlation information outside of the memory of subsequent decoder stages. 



Problem 10.36 



The basic idea behind the turbo principle is to use soft information from one stage as input to the 
next stage in an iterative fashion. For a joint demodulator/decoder, this could be arranged as 
shown in Fig. 1. 



Decoder extrinsic 




raw channel 
information 



soft-output 
demodulator 



BCJR 

2 



+ 



raw channel 
information 



soft-output 
decoder 



Demodulator 
extrinsic 



Figure 1 



In this figure, BCJR 1 is a MAP decoder corresponding to the Markov model of the modulator and 
channel; and BCJR 2 is a MAP decoder corresponding to the Markov model of the forward error 
correction code. The raw channel information is fed into the soft demodulator on the first 
iteration; this is combined with the extrinsic information from the previous decoding stage on 
subsequent iterations. The extrinsic information from the soft-output demodulation stage plus the 
raw channel information is the input to the decoding stage. Feeding back the extrinsic information 
from the latter stage closes the loop. At any stage the output from the decoder can be used to 
estimate the data. (Figure 1 shows a symmetric implementation. Other arrangements are 
possible.) 
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Problem 10.37 



Matlab codes 

*/. Probelm 10.37 , CS : Haykin 
'/. Turbo coding 
'/.M. Sellathurai 

clear all 

'/, Block size 

block_size = 400; '/, 200 and 400 

'/, Convolutional code polynomial 
code_polynomial = [ill; 101]; 
[n,K]=size(code_polynomial) ; 
m=K-l; 

'/, Code rate for punctured code 
code_rate = 1/2; 

'/, Number of iterations 
no_of .iterations = 5; 

'/, Number of blocks in error for termination 
block_error_limit = 15; 

'/. signal-to-noise-ratio in db 
SNRdb = [1] ; 
snr = 10~(SNRdb/10) ; 

'/. channel reliability value and variance of AWGN channel 
channel_reliability_value = 4*snr*code_rate ; 
noise_var = l/(2*code_rate*snr) ; 

'/.initializing the error counters 
block_number = 0; 

block_errors(l,l:no_of .iterations) = zeros(l, no_of .iterations) ; 
bit_errors(l, l:no_of .iterations) = zeros(l, no_of .iterations) ; 
total_errors=0 ; 

while block_errors(l, no.of _iterations)< block.error.limit 
block_number=block_number+l ; 

'/. Transmitter end 

'/. generating random data 
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Data = round(rand(l , block_size-m) ) ; 
'/, random scrambler 

[dummy, Alpha] = sort(rand(l,block_size)) ; 
'/. turbo-encorder output 

turbo_encoded = turbo_encorder( Data, code_polynomial , Alpha) ; 

'/, Receiver end 

'/. AWGN+turbo-encorder out put 

received_signal = turbo_encoded+sqrt(noise_var)*randn(l, (block_size) *2) ; 
'/, demultiplexing the signals 

demul_output = demultiplexer(received_signal, Alpha ); 
'/.scaled received signal 

Datar= demul_output *channel_reliability_ value/2; 

'/. Turbo decoder 

extrinsic = zeros (1, block.size); 
apriori = zeros(l, block_size); 

for iteration = 1: no_of .iterations 

'/, First decoder 

apriori (Alpha) = extrinsic; 

LLR = BCJLl(Datar(l, : ) , code_polynomial, apriori); 
extrinsic = LLR - 2*Datar (1 , 1 : 2 : 2*(block_size) ) - apriori; 

'/, Second decoder 

apriori = extrinsic(Alpha) ; 

LLR = BCJL2(Datar(2, : ) , code.polynomial , apriori); 
extrinsic = LLR - 2*Datar (2 , i : 2 : 2*(block_size) ) - apriori; 

'/. Hard decision of information bits 
Datahat (Alpha) = (sign(LLR)+i)/2; 

'/. Number of bit errors 

bit.errors ( iteration) = length(f ind (Datahat (1 : block_size-m) "=Data) ) ; 

'/. Number of block errors 

if bit_errors( iteration )>0 

block_errors(iteration) = block_errors( iteration) +1; 

end 

end 

'/.Total bit errors 

total_errors=total_errors+ bit_errors ; 
'/. bit error rate 
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if block_errors(no_of .iterations )==block_error_limit 
BER( i : no_of _iterat ions )= total_errors ( 1 : no_of _iterat ions ) / . 
block_number/(block_size-m) ; 

end 

end 
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function output = turbo_encorder( Data, code_g, Alpha) 

'/, Turbo code encorder 

7. Used in Problem 10.36, CS: Haykin 

'/.M. Sellathurai 

[n,K] = size(code_g) ; 
m = K - 1; 

block_s = length(Data) ; 

state = zeros(m,l); 
y=zeros(3,block_s+m) ; 

'/, encorder 1 
for i = 1: block_s+m 
if i <= block_s 

d_k = Data(l.i); 
elseif i > block_s 

d_k = rem( code_g(l,2:K)*state, 2 ); 

end 

a_k = rem( code_g(l, : )*[d_k ; state], 2 ); 
v_k = code_g(2, i)*a_k; 

for j = 2:K 

v_k = xor(v_k, code_g(2, j)*state(j-l)) ; 
end; 

state = [a_k;state(l:m-l)] ; 

y(l,i)=d_k; 

y(2,i)=v_k; 

end 

'/.encorder 2 

'/, interleaving the data 
for i = 1: block_s+m 

ytilde(l.i) = y(l,Alpha(i)) ; 

end 

state = zeros(m,l); 
'/, encorder 2 

for i = 1: block_s+m 
d_k = ytilde(i,i); 
a_k = rem( code_g(l, : )*[d_k ; state], 2 ); 
v_k = code_g(2, i)*a_k; 
for j = 2:K 

v_k = xor(v_k, code_g(2, j)*state(j-l)) ; 
end; 
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state = [a_k; stated :m-l)] ; 

y(3,i)=v_k; 

end 

'/. inserting odd and even parities 
for i=l: block_s+m 

output (i,n*i-l) = 2*y(l,i)-l; 
if rem(i,2) 

output(l,n*i) = 2*y(2,i)-l; 
else 

output(l,n*i) = 2*y(3,i)-l; 

end 

end 
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function [nxt_o, nxt_s, lst_o, lst_s] = cnc_trellis(code_g) ; 
'/.used in ProblemlO . 36 . 
'/, code trellis for RSC; 
'/. Mathini Sellathurai 



'/, code properties 
[n,K] = size(code_g) ; 
m = K - 1; 
no_of_states = 2~m; 

for s=l: no_of_states 
dec_cnt_s=s-l; i=l; 

'/, decimal to binary state 
while dec_cnt_s >=0 & i<=m 

bin_cnt_s(i) = rem( dec_cnt_s,2) ; 
dec_cnt_s = (dec_cnt_s- bin_cnt_s(i))/2; 
i=i+l; 

end 

bin_cnt_s=bin_cnt_s(m:-i: 1) ; 

'/, next state when input is 0 
d_k = 0; 

a_k = rem( code_g(l, : )*[o bin_cnt_s ]', 2 ); 
v_k = code_g(2, l)*a_k; 
for j = i:K-l 

v_k = xor(v_k, code_g(2, j+l)*bin_cnt_s(j ) ) ; 
end; 

nstateO = [a_k bin_cnt_s(l :m-l)] ; 
y_0 = [0 v_k] ; 



'/, next state when input is 1 
d_k = 1; 

a_k = rem( code_g(l, : )*[1 bin_cnt_s] ' , 2 ); 
v_k = code_g(2,l)*a_k; 
for j = 1:K-1 

v_k = xor(v_k, code_g(2, j+l)*bin_cnt_s( j ) ) ; 
end; 

nstatei = [a_k bin_cnt_s(l :m-l)] ; 
y_l=[l v_k]; 

'/, next output when input 0 1 
nxt_o(s, : ) = [y_0 y_l] ; 
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'/, binary to decimal state 
d=2.-(m-l:-l:0); 

dstateO=nstateO*d '+1 ; dstatel=nstatel*d'+l ; 
'/, next state when input 0 1 
nxt_s(s,:) = [ dstateO dstatei ]; 

'/. finding the possible previous state frm the trellis 
lst_s(nxt_s(s, 1), i)=s; 
lst_s(nxt_s(s, 2), 2)=s; 

lst_o(nxt_s(s, 1), 1:4) = nxt_o(s, 1:4) ; 
lst_o(nxt_s(s, 2), 1:4) = nxt_o(s, 1:4) ; 

end 
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function output = demultiplex(Data, Alph 
'/, demultiplexing the received signal 
'/. used in problem 10.36, CS: Haykin 
'/. Mathini Sellathurai 

block_s = fix(length(Data)/2); 
output=zeros(2,block_s) ; 

for i = 1: block_s 

Dataf(i) = Data(2*i-i) ; 
if rem(i,2)>0 
output(l,2*i) = Data(2*i); 
else 

output(2,2*i) = Data(2*i); 

end 

end 

for i = l:block_s 

output (1, 2* i-1) = Dataf(i); 
output (2, 2*i-i) = Dataf (Alpha(i)) ; 

end 
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function L = BCJLl(Datar, code_g .apriori) 
'/. log-BCJL (LOG-MAP algorithm) for decoder 1 
'/. Used in Problem 10.36, CS: Haykin 

'/, states, memory, constraint length and block size 

block.s = f ix(length(Datar)/2) ; 

[n,K] = size(code_g) ; 

m = K - 1 ; 

no_of_states = 2~m; 

infty = lelO; 

zero=le-300; 

'/, forward recursion 
alpha(l,l) = 0; 

alpha(l ,2 :no_of .states) = -infty*ones(l,no_of_states-l) ; 
*/. code-trellis 

[nxt_o, nxt_s, lst_o, lst_s] = cnc_trellis (code_g) ; 
nxt_o = 2*nxt_o-l; 
lst_o = 2*lst_o-l; 

for i = l:block_s 

for cnt_s = 1 :no_of .states 

branch = -infty*ones(l,no_of_states) ; 
branch(lst_s(cnt_s,l)) = -Datar(2*i-1)+Datar (2*i) * . . . 
lst_o(cnt_s , 2)-log( l+exp(apriori(i) ) ) ; 
branch(lst_s(cnt_s,2)) = Datar(2*i-1)+Datar(2*i)* . . . 
1st _o ( cnt_s , 4) +aprior i ( i ) -log ( 1+exp (apriori ( i ) ) ) ; 
if (sum(exp(branch+alpha(i, : )))>zero) 

alpha(i+l,cnt_s) = log( sum( exp( branch+alpha(i ,:)))) ; 

else 

alpha(i+l,cnt_s) =-l*infty ; 
end 

end 

alpha_max(i+i) = max ( alpha ( i+ 1, :)) ; 
alpha(i+l,:) = alpha(i+l,:) - alpha_max(i+l) ; 

end 

*/, backward recursion 
beta(block_s , 1)=0; 

beta(block_s,2:no_of_states) = -infty*ones(l,no_of _states-l) ; 

for i = block_s-l:-l: 1 

for cnt_s = l:no_of_states 
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branch = -infty*ones(l,no_of_states) ; 

branch(nxt_s(cnt_s,i)) = -Datar(2*i+l)+Datar(2*i+2)* . . . 
nxt_o(cnt_s,2)-log(l+exp(apriori(i+l))) ; 
branch(nxt_s(cnt_s,2)) = Datar(2*i+l)+Datar(2*i+2)* . . . 
nxt_o ( cnt_s , 4) +apr ior i ( i+ i ) -log ( 1+exp (apr ior i ( i+1 ) ) ) ; 
if (sum(exp(branch+beta(i+l, : )))<zero) 

beta(i,cnt_s)=-infty; 
else 

beta(i,cnt_s) = log(sum(exp(branch+beta(i+l ,:)))) ; 

end 
end 

beta(i,:) = beta(i,:) - alpha_max(i+l) ; 
end 



for k = l:block_s 

for cnt_s = l:no_of_states 

branchO = -Datar(2*k-l)+Datar(2*k)*lst_o(cnt_s , 2)-log(l+exp(apriori(k) ) ) ; 
branchl = Datar(2*k-l)+Datar(2*k)*lst_o(cnt_s ,4)+apriori(k)-log(l+exp(apriori(k) ) ) ; 
den(cnt_s) = exp( alpha(k,lst_s(cnt_s,l))+branchO+ beta(k, cnt_s) ) ; 
num(cnt_s) = exp( alpha(k,lst_s(cnt_s ,2) )+branchl+ beta(k,cnt_s)) ; 

end 

L(k) = log(sum(num)) - log(sum(den) ) ; 
end 
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function L = BCJLl(Datar, code_g .apriori) 
'/. log-BCJL (LOG-MAP algorithm) for decoder 1 
*/. Used in Problem 10.36, CS: Haykin 

'/. states, memory, constraint length and block size 

block.s = fix(length(Datar)/2); 

[n,K] = size(code_g) ; 

m = K - i; 

no_of_states = 2~m; 

infty = leiO; 

zero=le-300; 

'/. forward recursion 
alpha(i,i) = 0; 

alpha(l, 2 :no_of .states) = -infty*ones(l,no_of_states-l) ; 
'/, code-trellis 

[nxt_o, nxt_s, lst_o, lst_s] = cnc_trellis (code_g) ; 
nxt_o = 2*nxt_o-l; 
lst_o = 2*lst_o-l; 

for i = l:block_s 

for cnt_s = 1 :no_of .states 

branch = -infty*ones(i,no_of_states) ; 
branch(lst_s(cnt_s,l)) = -Datar(2*i-i)+Datar(2*i)* . . . 
lst_o(cnt_s,2)-log(l+exp(apriori(i))) ; 
branch(lst_s(cnt_s,2)) = Datar(2*i-1)+Datar(2*i)* . . . 
lst_o(cnt_s,4)+apriori(i)-log(l+exp(apriori(i))) ; 
if (sum(exp(branch+alpha(i, : )))>zero) 

alpha(i+l,cnt_s) = log( sum( exp( branch+alpha(i ,:)))) ; 

else 

alpha(i+l,cnt_s) =-l*infty; 
end 

end 

alpha_max(i+l) = max(alpha(i+l, : )) ; 
alpha(i+l,:) = alpha(i+l,:) - alpha_max(i+l) ; 

end 

'/. backward recursion 
beta(block_s.i)=0; 

beta(block_s,2:no_of_states) = -infty*ones(l,no_of_states-l) ; 

for i = block_s-l:-i: 1 

for cnt_s = 1 :no_of .states 
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branch = -infty*ones(l,no_of_states) ; 

branch(nxt_s(cnt_s,l)) = -Datar(2*i+l)+Datar(2*i+2)+ . . . 
nxt_o(cnt_s,2)-log(l+exp(apriori(i+i))); 
branch(nxt_s(cnt_s,2)) = Datar(2*i+l)+Datar(2*i+2) * . . . 
nxt_o ( cnt_s , 4) +apr ior i ( i+1 ) -log ( i+exp (apr ior i ( i+1 ) ) ) ; 
if ( sum ( exp ( branch+bet a ( i+ 1 , : ) ) ) <zero ) 

beta(i,cnt_s)=-infty; 
else 

beta(i,cnt_s) = log(sum(exp(branch+beta(i+l ,:)))) ; 

end 
end 

beta(i,:) = beta(i,:) - alpha_max(i+l) ; 
end 

for k = l:block_s 

for cnt_s = l:no_of .states 

branchO = -Datar(2*k-l)+Datar(2*k)*lst_o(cnt_s,2)-log(l+exp(apriori(k))) ; 
branchl = Datar(2*k-l)+Datar(2*k)*lst_o(cnt_s,4)+apriori(k)-log(l+exp(apriori(k))); 
den(cnt_s) = exp( alpha(k,lst_s(cnt_s,l))+branchO+ beta(k, cnt_s) ) ; 
num(cnt_s) = exp( alpha(k,lst_s(cnt_s,2))+branchl+ beta(k, cnt_s) ) ; 

end 

L(k) = log(sum(num)) - log(sum(den) ) ; 
end 
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Answer to Problem 10.37 
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